Red

News

Red/System v0.2.1 released

This new release is bringing a lot of important bug fixes and some new features for Red/System. The new PRINT function might break some existing scripts, but should be easily fixable.

New features

  • Variable-arguments function support
  • RTTI function limited support
  • Polymorphic and variadic PRINT function
  • Command-line arguments access
  • Added stack low-level manipulation native functions: push, pop
  • Access to stack pointer and stack frame pointer (reading/writing)
  • Subtracting two pointers is now possible
  • Preprocessor improved to solve macros recursively
  • New built-in hexdump debugging functions
  • External library access unit tests (thanks to Peter)

PRINT function examples:

print 123 
123 

print "hello" 
hello 

print [123 "hello"] 
123hello 

print [123 tab "hello" lf "world" lf] 
123    hello 
world 

a: 1 < 2 
b: "hello" 
print [a tab b/2] 
true    e

Bugfixes

  • Complex arithmetic expressions involving pointers, path access and type casting are now much more reliable. 89 new unit tests were written to help fix the bugs and track regressions.
  • Complete list of fixed issues available in Github’s tracker.

Specification document

  • Updated to match all new and changed features

Red runtime

  • Memory allocator implemented in Red/System, documentation is pending.

As you can see, no vacation this summer for Red project!

Red v0.3.1: functions support added

This new milestone brings (finally) functions to Red along with other new features and fixes. Let’s first have a quick overview of 0.3.1 release content:

  • function support
  • path get/set notation support
  • refinement support for native functions
  • expressions in parentheses compilation
  • new datatypes: function!, paren!, path!, lit-path!, get-path!, set-path!
  • new actions and natives: find, select, copy, reflect, type’, halt,…
  • extended mold, form and comparison operators to all new and existing datatypes
  • many new mezzanines functions
  • modulo and remainder operators on floats implemented for ARM backend
  • Quick-Test testing framework ported to Red
  • a truckload of new unit tests
  • many bugfixes
  • about 200 new commits to the Github repository
  • updated Red/System formal description document

Functions

They are declared using a similar syntax to REBOL, with some extensions. The specification block looks like this:

[arg1 [type1] arg2 [type2]...return: [type] /local var1 [type-v1]...]
  • Arguments and local variables are declared in the same way as in REBOL
  • A return value type can be optionally specified
  • Local words can have a type specifier

All type specifiers are optional, adding them will not only allow the compiler to make additional type checkings but also generate faster code in some cases (once the optimizations added to the Red compiler).

Note: argument and return value type checking have not been implemented yet, they need typeset! and error! datatypes to be implemented first.

Functions can be built using several constructors:

  • function: automatically collects local variables, like funct in REBOL.
twice: function [a [integer!] /one return: [integer!]][
    c: 2
    a: a * c
    either one [a + 1][a]
]
  • func: low-level function constructor, everything needs to be manually specified.
twice: func [a [integer!] /one return: [integer!] /local c][
   c: 2
   a: a * c
   either one [a + 1][a]
]
  • has: no argument, just lists local words.
globals: [1 2 3 _]

foobar: has [list][
    if list: find globals '_ [clear list]
]
  • does: no argument, no local words.
quit: does [
    print "Goodbye cruel world!"
    halt
]

Early exit points are also there in form of exit and return functions.

foobar: func [a [integer!] b [integer! none!]][
    if none' b [return none]
    if b = 0 [
        print "Error: division by zero"
        exit
    ]
    a / b
]

Refinements are also fully supported, when not used, their value is set to false (while in REBOL it is set to none).

Still some features are not yet implemented:

  • building a function at runtime (requires the ability to JIT-compile source code)
  • passing a function as argument (pending)
  • tail recursion optimization

Currently functions are using a local context on stack, so their life-time is very short. For indefinite extent support, a closure! type will be added in the next months.

Path support

Path datatypes were added (path!, lit-path!, get-path!, set-path!) to the runtime library and compiler, allowing the use of path notation as syntactic sugar for series access. Getting and setting values using path notation are supported.

Some usage examples:

list: [a 5 b [c 8] d #"e" name: "John"]

probe list/2        ;-- outputs 5
probe list/7        ;-- outputs name:
probe list/a        ;-- outputs 5
probe list/4/2      ;-- outputs 8
probe list/b/c      ;-- outputs 8
probe list/name     ;-- outputs "John"

list/b/c: 0
probe list/b/c      ;-- outputs 0

index: 3
probe list/:index   ;-- outputs b
list/:index: 'z
probe list/:index   ;-- outputs z

Note: notice how words lookups in blocks can work with any word datatypes.

Paren expressions

Paren! datatype and expressions in parentheses compilation support has been added too. They are mainly useful with infix operators in order to force a given execution priority. Example:

print (1 + 2) * (3 + 4) ;-- outputs 21

New built-in functions

  • actions: find, select, copy, reflect
  • natives: type', halt, function, func, does, has, exit, return
  • mezzanines: probe, first, second, third, fourth, fifth, last, spec-of, body-of and all <datatype>’ type testing functions.

Existing native or action functions were extended with refinements (like mold and form). See the boot.red script for more details.

Testing

The Quick-Test framework from Peter WA Wood was ported to Red and now powers all the current 1800+ unit tests written in Red itself, mostly added by Peter.

About 32 issues/bugs in both Red and Red/System were fixed in this new release.

Red/System formal description

Last but not least, Rudolf updated his Red/System formal description document [PDF, 162KB] to include the additional features like namespaces. Many thanks to him for his remarkable work and support to Red project!

What’s next’

Objects support is next priority along with new natives like switch and case. Typesets and error handling will probably be added very soon too. Once that part done, the next focus will be to add I/O support!

Many thanks to all the people that have made significant donations in last months to Red project, allowing me to continue to work full time on it! It is really making a big difference, as shown by Red increased progress speed since this summer. Thanks again!

Cheers!

Update on Red/System progress

Here is a quick update on Red/System progress.

Specifications Draft

Draft has gone through several revisions. Several decisions mainly regarding pointers handling have been taken with simplification and disambiguation as main goals:

  • Struct! values are now passed by reference (a syntax for passing by value will be added in future), simple arithmetic (+, -) can be used on struct references.
  • String! was renamed to c-string! to avoid possible future collision with a Unicode-aware string! type. However, this case is not totally solved, there are still some possible issues, like the literal syntax. As c-string! values are passed by reference, simple arithmetic was added too.
  • Pointer! datatype was restricted to point to integer! values only. It is more consistent with new c-string! and struct! which are both (implicit) pointers already, and that removes the need for verbose pointer syntax.
  • Array! datatype: no decision taken for now, we will see if (both explicit and implicit) pointer arithmetic support will be sufficient during Red’s runtime implementation.
  • Logic! datatype was added as a first class type. This makes boolean literal values (true, false) and conditional expressions assignable to variables and can be passed as argument or returned by user functions. Logical NOT operator was also added as well.
  • Byte! datatype (unsigned byte value) was added to ease string byte accesses.
  • Limited type inference was added for function’s local variable and return value types.

A couple of things to keep in mind about Red/System design:

  • during the implementation of these specifications, some better design options might appear in light of the implementation decisions.
  • at some point, when the bootstrapping phase will be over, Red/System will be rewritten entirely in Red, giving an opportunity to fix and improve the design (also helped by several months of feedback from Red/System usage).

Implementation Progress

Several items from the todo-list have been implemented in the last week, the most notable ones being:

Among other changes:

  • ELF emitter has been extended a bit to support sections table (thanks to Andreas).
  • An experimental Mach-O (OS X) support is under work by Rebolek.
  • Several unit test files have been added to the repository recently, using a temporary minimalistic approach, that will be soon superseded by a real dedicated framework.
  • We are close to have a nice, small and lean unit test framework (Quick Test), created by Peter W A Wood for Red/System, covering all our needs (internal and external testing).

An interesting side note on how implementation can influence back the design:

During the implementation of Quick Test framework, Peter, while writing an integer to string conversion function, came up with a code pattern that is common in REBOL, but wasn’t planned for Red/System: declaring private functions inside a function to hide them from global context. The pattern looked like this:

prin-int: func [...][
    ...
    prin-digit: func [...][...]
    ...
    prin-digit ...
    ...
]

I was surprised when I tested the code he sent me, as I never did anything in the implementation to support such case explicitly. It was working because of a side-effect caused by the way functions are compiled in Red/System: the functions bodies are all gathered and compiled at the end of the global context code, regardless of where they are defined. The side-effect making the inner “prin-digit” function private is caused by a global function variable being declared inside a local scope, but without being part of the function specification block. So, once the outer function compiled, the “prin-digit” symbol is lost, making the function unreachable from global context and forever hidden.

I was thinking about adding a way for users to create private contexts much later in the future (relying on Red/System source header), but we have the opportunity to have an equivalent feature, right now and without adding anything to the code! So, I am giving it some time to see if the side-effect can be safely made permanent to keep that feature, and in the meantime, added it to the Possible Evolutions part of the specifications.

In the next days, I will work on:

  • finishing implementing the features from the todo-list
  • merging the unit tests framework once ready
  • adding new unit tests

In my next blog article, I will give an overview of Red/System compiler internals as requested by some followers.

Posts:

Tags: