Philosophy and Principles

[segeljakt]

This is a list of principles for designing and implementing arc-script. Let me know if you think any principles are wrong or missing.

Design Principles

• Safety: The language should be safe in the sense that a well-formed program should be well-behaved.
• Loadable Code: Some code may not be known to the compiler.
• Memory Safety: Memory is automatically managed to prevent leakage and false dereferencing. This requires garbage collection, either at runtime (reference counting), or compile-time (linear types).
• Type Safety: Do not allow "trust me, I know what I'm doing" style of programming.
• Predictability: The language should have no "unwritten rules" that lead to unpredictable program behaviour.
• Static and Strong Typing: All terms must be well-typed at compile-time (inferred or annotated).
• Type Inference: The language should require minimal type annotations, preferably through global type inference.
• Context Free Grammar: Syntax should be separated from semantics to ease understanding.
• Data Abstraction: Data types should reflect the problem being solved.
• Polymorphism: Constructs can operate over different data abstractions which share the same interface.
• Creative Extension Principle: If non-local code modifications are needed to solve a problem, then a new concept should be introduced instead to handle that problem.
• Less is more: There should be a small set of primitives in the core language. Fewer constructs make the language easier to reason about.
• Separation of Concerns: Decisions about the design of Arc-Script should not be intermixed with decisions about its implementation.
• Multi-Layered Design: The language should have multiple layers which each offer a special trade-off of expressiveness and ease of reasoning.
• Orthogonality: The language should have an expressive surface syntax with syntactic abstractions that desugar into a minimal orthogonal kernel syntax.
• Formality: All language components should be formalised. This property is important if the language is going to be 1) re-implemented 2) proven to satisfy properties such as soundness, and 3) understood by its practitioners.
• Domain Specificity vs. Generality: The language should focus on being expressive specifically towards continuous and deep analytics. Some generality is also needed or else the language becomes too restrictive.
• Extensibility: Constructs not available in the language should be importable from the outside as Abstract Data Types.
• Productivity: The language should be productive to use. This means it must be possible to reuse code which others have already written.
• Uniformity: Language constructs should be consistent with respect to each other in terms of both appearance and behaviour, "the syntax should make the semantics clear".
• Regularity: Standard language constructs should be consistent with respect to accepted notations and conventions.
• Debugability: Language violations should be caught and reported back to the user, even if they occur at runtime. In other words, there must be no uncaught and undefined behaviour. Error messages should also be sensible.
• Implementation Independence: Semantics should be implementation-independent with respect to compiler or interpreter.
• Readability over Writeability: Code is read more times than it is written. Favour making the language easy to read, and provide editor support (such as completions) for making it easy to write.
• Scalability: The language should have support for building scalable solutions. In particular, it should be possible to build massive pipelines without loss in performance, and to re-use operators by making them polymorphic and exporting them as modules.
• Transparency: How the runtime system works should be transparent to the user. For example, management of progress and parallelism should be invisible.
• Power users: Users who desire more flexibility, to for example fine-tune performance, should be able to program Arc-Scripts using lower level primitives than those which are provided out of the box. This can be made possible by exposing an interface at the Rust-level.
  • The motivation for not having this interface in the language is it is very important that the language does break properties such as event time. With low-level primitives (such as channels), event time cannot be made transparent without the possibility of violating it, and therefore must be made explicit. The language is however not meant for implementing systems, but instead meant for implementing applications running on a system. Therefore low-level concepts are only exposed at the Rust-level.

Implementation Principles

• Code reuse: Reuse existing frameworks when implementing the compiler to save time.
• Demo Driven Development: Before implementing new features, create a demo to motivate why the feature is needed.

[frej]

Separation of Concerns: Decisions about the design of Arc-Script should not be intermixed with decisions about its implementation.

This allows for language features which can't be implemented or would penalize the performance of other language constructs.

[segeljakt]

This allows for language features which can't be implemented or would penalize the performance of other language constructs.

At the extreme the language could either be designed bottom-up (prioritising the needs of the machine) or top-down (prioritising the needs of the programmer). C++/Rust are bottom-up and Python/Erlang are top-down. I think we should prioritise the programmer and take what you say into consideration. Some language features are going to be tough to implement efficiently in Rust, at least without major compiler hacks, so maybe it is best to prioritise the features which are easy to implement 🤔

[segeljakt]

Added a point about Power users

[segeljakt]

Added a point about Demo policy

[frej]

Added a point about Demo policy

DDD -- Demo Driven Development :)