Paste: clay todo

Hi Guys,

This is a list of Clay sub-projects which need work. Please add your thoughts and comments if any. Also, if you think of any other sub-project which I've missed out, please add that too.

- KS

Miscellaneous pending features (KS)

(DONE: 25/01/10) improve module system
import module;
import module as foo;
import module.*;
import module.(foo, bar);
import module.(foo as foo2, bar, baz as baz2);
(DONE: 26/02/10) direct return value access
(DONE: 26/02/10) make types overloadable to define custom constructors
(DONE: 27/02/10) make records overloadable to define custom pattern-matching constructors
(DONE: 28/02/10) static globals, static procedures, and static overloadables
(DONE: 28/02/10) added TypeOf primitive
(DONE: 28/02/10) overloadable dereference operator
(DONE: 28/02/10) overloadable indexing operator
(DONE: 01/03/10) add update assignment operators
(DONE: 02/03/10) overloadable call operator
(DONE: 05/03/10) lambda
(DONE: 05/03/10) static objects such as procedures, overloadables, records, and types should be usable as values
(DONE: 07/03/10) enums
(DONE: 08/03/10) global variables
(DONE: 08/03/10) public/private definitions in a module
(DONE: 09/03/10) external C variables
(DONE: 20/03/10) dlls
(DONE: 21/03/10) 64-bit support
(DONE: 30/03/10) bindings generator
(DONE: 31/03/10) libc wrappers for mac, linux, and windows
(DONE: 01/04/10) win32 dllimport, dllexport attribute for externals.
(DONE: 01/04/10) win32 stdcall calling convention for externals
(DONE: 05/04/10) inlined procedures and overloadables (call-by-name semantics)
(DONE: 06/04/10) overloading based on lvalue/rvalue
(DONE: 10/04/10) variable arity procedures and overloads
(DONE: 10/04/10) LValue? primitive
(DONE: 10/04/10) sequences library
(DONE: 15/04/10) I/O library
(DONE: 21/04/10) api for memory mapped files
(DONE: 26/04/10) 'new' operator
(DONE: 29/04/10) data structures in memory mapped files
DONE: implement codecontext parameter for analyze, eval, codegen
DONE: cleanup return by ref syntax, remove returnref statement.
DONE: add return-by-ref support for lambdas, blocks
explicit naming of return values. drop 'returned' keyword.
explicit naming of variadic arguments.
unpacking of tuples and arrays.
static expressions.
make static arguments just syntactic sugar
cleanup invoke lookup process
add multiple values support
add cmdline flags -L -l -c
conditional compilation with 'static if'.
improve literal suffixes
identifier literals
computed records
remove built-in pattern matching constructor for template records
allow arbitrary type expressions as record field types
implicit constructor return type. (verify if explicit)
make generic String concept
block expression
if expression
overloadable field-ref operator
'truth' overloadable, to allow arbitrary types in if/while conditions.
add something equivalent to java's 'super' for overloads
unions for dynamic dispatch
callback closures

Light-weight threads

Should be scalable to multiple cores.
Should not have locking penalty for single-core execution.
Ideally the stack should be growable. (but not necessary immediately).
The stack size should be a parameter during thread creation.
Should work on Mac OS X, Linux, and Windows.
A C-level library for this is almost ready.
What should be the clay level API?


File and Network I/O

Clay level API should be blocking, but implement using non-blocking calls internally.
Make it play well with threads.


Exceptions

Adding exceptions to a non garbage collected langauge is very hard and tricky. Thankfully, it has already been for C++. There is a perception that exceptions are hard to use in C++. I believe that this perception is only due to historical reasons. Modern C++ with smart pointers and exception handling can be used to write large software with almost no manual memory/resource management required. The best source for understanding exception handling in c++ is Herb Sutter's book(s) titled "Exceptional C++".
The C++ exception handling semantics should directly be used in Clay.
The exception value itself will be of type 'Any'.
What should be the design and syntax of the 'try', 'catch', and 'finally' statements.
In the event of an uncaught exception the thread throwing the exception is exited and exception thrown is printed to stderr. I think we should also show file name and line number.

Sequences library

Model this on the 'Range' idea from the D programming language.
Lot of inspiration can be taken from lazy lists in haskell too.
See article by Andrei Alexandrescu titled "On Iteration" - http://www.informit.com/articles/printerfriendly.aspx?p=1407357
See presentation by Andrei Alexandrescu titled "Iterators must go".
Slides - http://www.boostcon.com/site-media/var/sphene/sphwiki/attachment/2009/05/08/iterators-must-go.pdf
Reddit discussion - http://www.reddit.com/r/programming/comments/976aj/iterators_must_go_presentation_by_andrei/

Hashtable library

Study good hash table implementations like those available in GNU C++ Lib, and Google's sparse hash.
Borrow ideas and implement something nice and simple for clay.

Auto-generation of C library bindings

Currently, 'external' declarations can be used to declare C functions in external libs which can be directly called from clay code. When interfacing with a large C library, having to declare all the functions from the library becomes a very large task. Investigate how we autogenerate the required declarations (perhaps by using clang).
How do we handle preprocessors definitions from external libs?
#defines used for constant values can be handled differently from general macros.
How do we handle struct definitions and enums?

Debugging Support

Writing our own (cross platform) debugger would take a lot of work. Easier thing might be to just add clay as a source language in GDB. Since clay is compiled to machine language, we will also have to generate DWARF/COFF debugging tables, which GDB understands. 
On Windows we can generate .PDB files, which would make us compatible with most debugging (WinDBG, Visual Studio Debugger) and profiling (AMD CodeAnalyst, VTune?) tools.