[Deji]

Project 'SCRambl'

Download Current BETA

View Project

Info
SCRambl is a high-level SCR language assembler with an advanced modular design which allows for a deep level of customisation. Compiled SCR scripts are often used by desktop and game software which read the file and execute the appropriate code internally, allowing an easy lower-level way of creating content. SCRambl supports both headered and headerless script compilations with a wide range of support for changing the way their code is generated. SCRambl also has many preprocessor features including macro support and preprocessor blocks which allow for improved code management and maintainability within large projects.

SCR
SCR is a conceptual language based on BASIC, which provides a fully featured scripting environment while maintaining a relatively low learning curve. This language is considered incomplete as many decisions have still not yet been made. SCRambl tries to allow for complete customisation of the SCR syntax through the use of definition files, enabling it to be easily updated, expanded or redone, so much as to allow SCRambl to be used for a wide range of coding styles, optimisation techniques and required usages.

Features

• Fast code parsing & compilation
• Advanced preprocessor
• High construct
• Easy syntax expansion & customisation
• Deeply customisable compilation
• Accurate default SCR syntax
• Fast hashing & lookups
• Syntax expandable from within scripts
• Macros, advanced preprocessor logic/integer expressions, preprocessor blocks
• Fully working string functionality including special character ability using escape sequences
• String escape sequences
• Error detection
• Useful warnings
• Variable scoping
• Configurable code optimisations
• Operators and command overloading
• Advanced configurable type matching and recognition system
• File including

Future Features

• Project support
• Internal scripting engine
• More customisability
• Performance optimisations
• Customisable header compilation

Examples
I started out by building up SCRambl to build a simple native script, debug.sc, which is found in SA's data folder. Once it got over compiling that, I started to make a dent on compiling some native GTA III scripts, which it got through quite well (though I had to remove III specific commands as I haven't built a catalogue of them yet) but I never finished parsing those fully. Eventually I started writing a short script which I am constantly adding and taking bits from to test individual features (test-script.txt) and decompiling them in Sanny Builder to verify the results.

Here are some of the more impressive abilities...

In-script Command Registering and Overloading

CODE

#register_command 0EEE MY_COMMAND(i,f,f)
MY_COMMAND(1, 0.5, 0.5)   // compiles 0EE command
#register_command 0FFF MY_COMMAND(f,f,f)
MY_COMMAND 3.0 1.2 3.2

SCRambl allows you to define new script-specific commands which is great if you do something like hacking in a new opcode using the script or have added new opcodes using another project-specific file which you don't want to keep globally or wish to share easily. Since the command is defined in the script, no one you give the script to will have to edit any of their configuration files in order to add new opcodes. The syntax of the preprocessor command "register_command" is much like the commands.ini syntax with a few restrictions such as not being able to define an unknown/NOP command.

You can then use a newly registered command after it has been defined or overload it. The first use of MY_COMMAND compiles the 0EEE opcode, because it uses an integer for the first parameter. The second use compiles the 0FFF opcode, because it has a float instead. SCRambl searches through command overloads to find the most fitting one for the job. If the 0EEE variation didn't exist, writing an integer as the first parameter would compile the 0FFF variation using an integer for the float. You can do the same if the 0FFF variation didn't exist. However, a warning will be produced saying that converting a float to int may result in a loss of data.

Native Control Structures

CODE

VAR_INT counter

REPEAT 5 counter
    WAIT 0
ENDREPEAT

Decompiled in SB3:

CODE

:NONAME_7
WAIT 0
$PLAYER_CHAR += 1
 $PLAYER_CHAR > 5
GOTO_IF_FALSE @NONAME_7

Script Constant Support + Local Variable Scope + Arrays + Enumerators + Long Math Lines (lol)

CODE

#const
    flval    FLOAT    4.0
#endconst

{
   LVAR_INT int_var int_arr[4] int_idx
   LVAR_FLOAT float_var
   int_arr[int_idx] = 69    // 1@(5@,4i) = 69
   int_var = TRUE    // 0@ = 1
   int_var += 5 + 4 + timera + 2 + 1   // >> /*
       0@ += 9
       005A: 0@ += 32@ // (int)
       0@ += 3
   */
   int_var -= timera    // 005A: 0@ += 32@ // (int)
   int_var -= PAD1    // usage of an enumerator as a value
   int_var = flval    // 0062: 0@ -= 32@ // (int) (causes float-to-int warning)
   float_var = flval    // 6@ = 4.0
}

Built-in Model Constants + Macro's

CODE

#define DEFAULT_WAIT_TIME (250 + 5)   // obviously DEFAULT_WAIT_TIME will be 255
#define INTERRUPT WAIT DEFAULT_WAIT_TIME   // now we can write "WAIT 255" using the macro "INTERRUPT"

#define SuperVar LVAR_STRING   // yes, SCRambl will be able to turn SuperVars into a full compiler extension

#if (defined(SuperVar)) + 2 == 3    // this returns true, so "WAIT 255" is produced instead of "WAIT 0"
    INTERRUPT
#else
    WAIT 0
#endif

INTERRUPT   // of course, there are much better ways to use macros...

SUPERVAR SVar    // guess what? we just declared a SuperVar! easy as 1, 10, 11... get it?

#define ADD_MALE_CHAR CREATE_CHAR DEFAULT_MALE_CHAR 0.0 0.0 0.0
CREATE_CHAR PEDTYPE_CIVMALE male01 0.0 0.0 0.0 male_char
ADD_MALE_CHAR male_char

Decompiled in SB3:

CODE

WAIT 255
WAIT 255
CREATE_CHAR 4 #MALE01 0.0 0.0 0.0 $PLAYER_ACTOR
CREATE_CHAR 4 #MALE01 0.0 0.0 0.0 $PLAYER_ACTOR

Pretty cool if I do say so myself...


The Manual
A manual is provided with each version of SCRambl, which documents the features in that version. The manual comes in HTML format, using JavaScript to make it easier to browse and read. This documentation has been tested working with the latest versions of Mozilla Firefox and Google Chrome.

An online copy of the documentation for latest version SCRambl can be found at: https://dl.dropboxusercontent.com/u/46693209/MANUAL.htm


Manual
View Project (changes)

[Deji]

A HTML documentation is available for SCRambl: https://dl.dropboxusercontent.com/u/46693209/MANUAL.htm
Brush up on your SCR and get your brain SCRambl'd!

EDIT: Added new 'Basics' section on strings - https://dl.dropboxusercontent.com/u/4669320...#Basics.Strings

This post has been edited by Deji: Sep 16 2013, 07:36 PM

[Deji]

Check the first post!!!!!

It's real!!!!!

[Sidewalk Surfer]

Fist thing I do when I get home from work is try this out!!

[Deji]

In case anyone wants to use Sanny Builder as the IDE for SCRambl, here's a little pic to show how it's done:

The user tools dialog is in Tools > IDE Tools > User Tools... Remember to click "Update" to update the highlighted user tool or "Add" to add it to the list. Set an easy shortcut and use it when you want to compile a SCR script. Or go Tools > IDE Tools > SCRambl...

The -p command line param at the end keeps the console window open after compilation or an error has occurred. It might be important to see when a compilation has failed due to a coding mistake.

In order to have syntax highlighting, it's recommended you use "SCR for Sanny" (v2 entries from the OpcodeDB) for now. The definition files that come with SCRambl are actually fairly outdated. These things will be improved as SCRambl breaks out of BETA and I can make some certain decisions on how to continue with certain aspects of the compiler, which is where your input will be much appreciated

[Deji]

SCRambl 1.0.35 (BETA) Released!

+ New directive #register_var for registering variables in-script with custom indexes
+ Added new field 'array size' for variables.def definitions, specifying variable array sizes
+ Added new field 'is timer' for variables.def definitions, specifying a variable is externally modified
+ Added handling of hashes instead of opcodes (partially)
* Changed the syntax of variables.def
* Improved performance (a lot)
* Improved operation matching
* Fixed some definition loading bugs
* Fixed operation matching bug where the parser would use loosely compatible opcodes for var assignments

[Deji]

Been working on some code to generate SCR definition files for Notepad++! Progress is good...


If all goes well, Notepad++ might be the first official UI choice for SCRambl, since it has pretty much everything we'll need...

EDIT
Another (bit of a tease)!

[Deji]

SCRambl 1.0.36 (BETA) Released!

* Updated command definitions in commands.def
* Fixed bug where SCRambl would produce an 'Unrecognized character escape sequence' warning when using the '\\' escape sequence
* Fixed bug where variable argument lists didn't work properly
* Fixed bug where trying to define 'variable return lists' threw an error
* Fixed bug where SCRambl wasn't throwing errors for undefined label usage
* Fixed bug where conditional command overloading didn't work properly
 
EDIT
Here are the files for Notepad++ SCR syntax. Follow the readme.txt instructions very carefully! It's still a WIP!
https://dl.dropboxusercontent.com/u/46693209/SCRambl-npp.zip

[Deji]

SCRambl 1.0.38 (BETA) Released!

I was in the too in the zone to release 1.0.37, so I managed to make this a huge update...

1.0.38
+ Added parsing of language INI files for customisable/translatable output
+ New directive #import for inclusion of various types of data and parsing of definition files
+ SCRambl now copies final compilations to the project (source script folder) directory
* SCRambl now searches for source files in contained "lib" folders, allowing for global or application-specific include files
* Improved file management
* Improved error handling
* Improved messages and output
1.0.37
+ New directive #error for throwing errors in the preprocessor and preventing further parsing
+ New directive #warning for outputting warnings in the preprocessor
+ New directive #register_type for defining new types in-script
+ Added casting ability, e.g. "lvar_int:32" to produce an LVAR_INT with index '32'
* Fixes to macros being used in-code
* Fixed preprocessor bug where #else wouldn't be processed after an #if evaluating to false
* Fixed crash using operators due to previous update
* Improved translation parsing (now accepts "LVAR_INT[VAR_INT]", non-array variable types handled like arrays with less automatic data available in translations)
* Improved type matching & command overloading
* Improved performance

SCRambl has gained some updates which make it friendlier with the concept of library scripts, and uses certain "lib" folders as additional #include (and #import) directories. This will allow easy extension of the scripting environment which can be game-specific, or even mod-specific (I'm working on SuperVars v2, now).

The #import directive, as well as being able to include binary files within a compilation, can load additional definition files, which are parsed the same way as the default ones loaded from each application definition folder, which is sure to come in useful for me.. I mean, us all.. I mean, whoever still mods

SCRambl also now uses language files for text output, so text is customisable/translatable. Unfortunately, my lack of knowledge of world languages means this is just a theoretically useful update, lol.

[DK22]

Can't find any reasons why

CODE

while not

can't be used same as

CODE

WHILE NOT

Any option for this?
Also, is there a quick way to get/store values from/into variables inside #asm construction?

In case anyone wants to use Sanny Builder as the IDE for SCRambl, here's a little pic to show how it's done:

Doesn't SB marks code in {} as commentary?

Also, is this the way how do I need to use "while (true)"?

CODE

#define TRUE IS_PC_VERSION

Main:
{
    LVAR_INT i
    WHILE TRUE
        WAIT 0
        i++
    ENDWHILE
}

Edit: I can't compile nothing with it. Even

CODE

#const
    flval    FLOAT    4.0
#endconst

Gives me error "unexpected string flval on line..."

This post has been edited by DK22: Dec 6 2013, 01:52 AM

[Deji]

Can't find any reasons why

CODE

while not

can't be used same as

CODE

WHILE NOT

Any option for this?

You're saying SCRambl's acting case-sensitive here? For me, even 'While Not' compiles...

Also, is there a quick way to get/store values from/into variables inside #asm construction?

LVAR's are converted to indexes, and VAR's to offsets, just like they would be elsewhere in code. The thing you need to write/read values would be their addresses, which also requires knowledge of the running script base. SCRambl can't predict the latter, so that's the part that needs to be taken care of during execution.

Also, is this the way how do I need to use "while (true)"?

It conflicts with the enumerator definition in constants.def:

CODE

BOOLEAN {
    FALSE
    TRUE
} b

OFF        FALSE
ON        TRUE

It's hard to explain, but "WHILE TRUE" would never syntactically work in SCR. SCR is designed around the script engine doing the logical work, not the compiler. A C++ compiler interprets "while(true)" as "while(1)", and one might even go as far as generating code looking like this (without optimisation getting in the way):

CODE

while:
mov al, 1
test al, al
jnz while

As you know, a condition result in C++ is simply returned from a function. C++ handles the return value directly after the call, wheras the logic is all handled internally when executing a SCM script, so SCRambl won't be able to optimise anything out. The natural way to implement a "WHILE TRUE"-style loop would be to use an actual command for the script engine to execute.

It would be better to create a different control structure to avoid screwing with R*'s original language too much. These are decisions I've been battling with for ages (so GTASA mobile is great news right now) and I've come to make many realisations about what separates languages and what goes into compiling them.

Edit: I can't compile nothing with it. Even

CODE

#const
    flval    FLOAT    4.0
#endconst

Gives me error "unexpected string flval on line..."

I think #const may be specifically bugged. What's more, I don't like it at all. I don't think it looks too good in use and has actually totally lost it's value, now that macros and casts exist. I'll probably be removing #const before pulling SCRambl out of BETA. Use #define instead as a more permanent solution.


I've got a lot of things to fix, but finally having some feedback has given me a few big ideas of things that need to change and be implemented sooner, so I'll dive into these things over the weekend. Customisable control structures will definitely be a priority.

Of course the more you let me know how you try to use SCRambl, the faster I can perfect the more important things

[Wesser]

Doesn't SB marks code in {} as commentary?

I never attached SCRambl on Sanny Builder, but I believe if you specify an external tool it will run at the compiling time, therefore you would use SB just as a sintax highlighter rather than a compiler.

It would be better to create a different control structure to avoid screwing with R*'s original language too much. These are decisions I've been battling with for ages (so GTASA mobile is great news right now) and I've come to make many realisations about what separates languages and what goes into compiling them.

We could hijack such syntactical limitation by implementing an user-made constract like LOOP-NEXT (ENDLOOP sounds like it breaks the constract prematurely rather than returning back to its beginning endlessly, while CONTINUE is reserved for skipping the code below and jumping to the next loop step even though not natively). This way we can give to the illusory WHILE-TRUE-ENDWHILE control flow a more likely idea of what it does.

I think #const may be specifically bugged. What's more, I don't like it at all. I don't think it looks too good in use and has actually totally lost it's value, now that macros and casts exist. I'll probably be removing #const before pulling SCRambl out of BETA. Use #define instead as a more permanent solution.

I agree for erasing it completely. You can overwrite it with #register_enum by allowing the same syntax of constants.def inside.

Oh, I forgot. Great work so far. Keep it up.

EDIT: I advice you to focus on improving SCRambl and leave me all the research part when the mobile version of San Andreas will be out, as happened till now I guess. III's information will come straight afterwards (they are quite similar to those of VC which have been already extrapolated).

This post has been edited by Wesser: Dec 7 2013, 03:24 PM

[DK22]

This works fine for me.

CODE

// This file was decompiled using sascm.ini published by GTAG (http://gtagmodding.com/opcode-database) on 14.6.2013
{$CLEO .cs}

//-------------MAIN---------------
thread 'Test'

:Test_11
if
0485:   return_true
else_jump @Test_49
wait 0
Actor.StorePos($PLAYER_ACTOR, 0@, 1@, 2@)
jump @Test_11

:Test_49
0A93: end_custom_thread

Any example for #asm #endasm? It doesn't work for me.

"Unexpected string 'fld' on line 12".

This post has been edited by DK22: Dec 7 2013, 08:07 AM

[Deji]

Okay, fixed the ASM problem and a few other minor bugs...
Hotfix: https://dl.dropboxusercontent.com/u/4669320...38.1-hotfix.zip

[DK22]

"Unknown char in identifier ',' on line 14".
After I removed all ',' from asm code, it gave me
"'Sqrt' : expected operand (arg3) on line 26".

[Deji]

I'll fix the other ASM issue, but in the other code, unless you edited commands.def, the return var should come last...

CODE

0AC6 GET_LABEL_POINTER(p,=i)

e.g.

CODE

GET_LABEL_POINTER(Sqrt) func_sqrt

SCRambl sees a variable as the first token of the line, followed by the '=' operator. Which begins an assignment of the index of command 'GET_LABEL_POINTER' (2758) to the variable, so it assumes 'Sqrt' to be part of the operation.

I like to use the brackets (which are ignored in SCR) to prompt Notepad++'s arg list, but then leave return vars outside of this argument list. That's just my preference, but it does make it easier to differentiate between ingoing and outgoing data.

Same with 'CALL'. Return comes last, since it's based on the order these params are retrieved and sent by the script engine.

EDIT: Updated hotfix.

[DK22]

"ERROR: Still in ASM structure at end-of-file on line 36".

[Deji]

D'oh... fixed.

EDIT
While SCRambl is so far just an external script compiler, I think I will start working on better ways to implement the CLEO library in scripts, e.g. a single include will load all required parts of the CLEO Library:

CODE

#include "cleo.sch"

cleo.sch:

CODE

#import TYPES "cleo\\cleo3-types.def"   // import types such as audiostream handles, etc.
#import CONSTANTS "cleo\\cleo3-constants.def"   // virtual keycodes, important values
#import COMMANDS "cleo\\cleo3.def"

#if !defined(CLEO_LIB_LEGACY)
    #import TYPES "cleo\\cleo4-types.def"
    #import CONSTANTS "cleo\\cleo4-constants.def"
    #import COMMANDS "cleo\\cleo4.def"
#endif

#if !defined(CLEO_LIB_CODELESS)
    #hex
    00 00
    #endhex
#endif

Gotta think up the specifics. Also the CLEO 4 IntOperations.cleo library needs additional opcodes for proper SCR-style operations with only 2 parameters, so operators can be added for them in SCRambl.

Still could do with the OpcodeDB for the proper command definitions but I've still got to upgrade the param information for a couple thousand more entries

[DK22]

CODE

#define SOME_FUNC 0xFFFFFF

SCRIPT_NAME( "Test" )

CALL( Main , 0 )
TERMINATE_THIS_CUSTOM_SCRIPT( )

Main:
{
    LVAR_INT result
    CALL_FUNCTION_RETURN SOME_FUNC 0 0 result
    RET 0
}

"ERROR: 'result' : ????????? (arg 6) on line 11"

This post has been edited by DK22: Dec 8 2013, 05:57 PM

[Deji]

That's a definitions problem. The commands.def file was generated from the OpcodeDB around the time I was just starting SCRambl by converting the SB output to kind of fit. It's only been updated by some search & replace operations as I've changed certain things in SCRambl, so nothing has been worked on by hand much.

I think, in commands.def you just need to search for and replace this:

CODE

0AA7 CALL_FUNCTION_RETURN(i,i,i,+*,=*,0) ; isn't valid, since we have a limitless var arg list, followed by another

with this:

CODE

0AA7 CALL_FUNCTION_RETURN(i,i,i,+*,0) ; expect 3 integers, a mixed-type var-arg list, end with null byte

Unfortunately, since the "pop" parameter get inbetween the num_params arg and the arg list, this won't be able to take advantage of SCRambl's ability to use the number of params specified in-script to distinguish between the arg list and the returns, unless you abuse the pop parameter and replace with this instead:

CODE

0AA7 CALL_FUNCTION_RETURN(i,i,#*,+=*,0) ; expect 2 integers, an integer followed by that number of mixed-type args, a mixed-type var-arg return list

The OpcodeDB will be able to generate some better definitions in some time.