Complete null safety

Embarrassingly, it took me almost seven years to notice the full meaning of Option/Nullable type combined with mechanism like pattern matching.

The first layer is pretty obvious — consider such language as C# and its struct type. It is totally safe to use because it does not allow null. The separate world is nullable struct, here you can use null but your code is safer anyway because there is distinction between pure struct instance and nullable one. Until you write:

nullable_printer.Value.Print();

You pass null and boom, you have exception in your hands. The problem in this case is you can access the internals of nullable type.

Skila does not have pattern matching, but similarly it treats Option as black box — you cannot dereference it, you cannot access any of its member, the one thing you can do is decompose it. Conditionally:

if let printer ?= nullable_printer then
  printer.Print();
end

The optional declaration is successful only when we don’t have null. This way working with null is completely safe because compiler guarantees the logic flow is solid and you don‘t see any NullException. One thing less to worry, so I call it a progress.

Advertisement

Logical operators for null

Choosing the way of handling null with logical operators is basically a choice between three valued logic or typical “pragmatic” treating not-true values as false. I like the former approach but I chose the latter — it is faster to implement and run, probably it will be easier to grasp and most importantly I don’t see benefits of three valued logic. Every if branch is executed or not, there is no a third way for it, so if I ultimately ask “is this true” I can ask the same question right away when I see null.

Well, if I missed something here in worst case I will change the null logic.

Into the valley of nulls

Working on the new design of null brought so many challenges that I didn’t implement any special support for them in generic types — the concept is pretty alien to me and I don’t know what to fix because nothing is broken yet.

Skila features stackable null values with no performance penalty compared to regular values. It means indexOf for String does not use magic value -1 any longer, the pair first/firstOrDefault of the collection is the pattern of the past. All you need to return is optional value.

You can work with optional values safely:

let i ?Int = null;
test(i); // compile error

def test(i Int) Void ...

Can you guess why you will get compile error? Type mismatch? Close, but not enough — types do match here but conditionally (there is precisely one level of Option<T> difference) and this means the entire call is executed conditionally as well. Statement became expression with the outcome failure or success which has to be read or explicitly dropped. Like here:

let i ?Int = null;
_ = test(i); 

def test(i Int) Void ...

For the record there is no magic here, just regular error:

let i ??Int = null;
test(i); // classic type mismatch

def test(i Int) Void ...

There was no need to introduce special syntax for optional chaining (besides there would be asymmetry between optional caller and optional argument), so instead of clunky “?.” you write:

var s ?String = null;
s = s.toLower(); 

which is equivalent of:

var s ?String = null;
s = s is null ? null : s'value.toLower(); 

Unlike pre-6.0 C# no exception is thrown, unlike C# 6.0 the syntax is clear, and unlike Icon you cannot simply ignore the optional execution. Time will tell whether such approach is indeed a winner.

Leaving the trenches of the Option<T> type

Please note currently Skila has only reference types (not that it is well thought over decision, I simply don’t have enough time to deal with value types).

The problem how to return data from the function efficiently while fitting in special “no result” value, haunts me for some time. I am aware of several approaches:

1. magic values, like -1 for indices (it is still used, see Go),
2. thin Nullable<T> type to mark there is no result (you can see this in Kotlin),
3. rich Option<T> type for wrapping the actual value (Swift, Scala approach),
4. Try-Parse pattern (for sure it is used in C# libraries).

The first approach is dead simple and fast, yet it is disaster when it comes to serious programming — each time you have to check what magic value you can get. The second approach is much better but it does not scale very well — you can use it for a substring lookup, but it fails with dictionary getter or such functions as first on collection. Yet, the performance is on par with magic values.

For me the only realistic answers are the last two — with Option<T> type you can work with any function, you just add an extra layer of Option<T> to the working type. The performance suffers but you always have fast counterpart functions tryDoSomething. Of course some poor soul has to write all those pairs of functions now — because of that I was seriously considering supporting two types at the same time, lightweight Nullable<T> and rich Option<T>.

Oh yes, there is another approach:

5. communicate failure (like in Icon),

I have no idea what the performance is, but the Icon code is simply beautiful. It was about time to read a book about it — I barely even opened The Implementation of the Icon Programming Language by Ralph E. Griswold and Madge T. Griswold when I found this inspiring passage:

(…) the design of programming languages should not be overly inhibited by perceived implementation problems, since new implementation techniques often can be devised to solve such problems effectively and efficiently.

I am sold — I want beauty, and I want performance! Nothing less. I want stackable (nested) Option<T> type, easy to use with speed of raw null values. Until now I was focusing on optimizing nested Option<T> type, but maybe I could somehow add a stack to null… wait a second.

Let’s consider what happens on the first nesting level (think of Option<T>) — we have either the actual value or a null. On the second level (Option<Option<T>>) — the actual value or a null again. At both levels when we have the actual value we can recognize that we didn’t end up with no result because our reference is not equal to null. It is the null which has to be wrapped (because null from the first level of nesting plays a role of true data on the second level), the real, actual value does not need any wrapping. It is some progress but we still have to do a little wrapping, right? No, we don’t — just turn the microscope on and take a look. In the first case the failure is indicated by null¹, in the second case by another, different null — null².

Click, click, click — do you hear this sound?

Option<T> type. Who said it is a regular type in the first place? It does not have to be — our Option<Option<T>> is a disjoint union of types. It is Null² type, or Null¹ type, or T type — Null²|Null¹|T.

Because we have to tell compiler what type we would like to get, it knows at what level it operates — i.e. how many layers it should pretend unwrapping to get the value. If it is any of the null value cases, it will also know how long it should pretend it has real data — the show with single null keyword is just for the user. Say the pointer is set to null¹ and our current type is Option<Option<String>> — do we have real value? Sure thing, it is not null² and that’s all we have to care about.

Thanks to all those lies there is no wrapping values in the runtime, the speed is the same as working with plain old null values. The only difference comparing to rich Option<T> school is with down casting — we will be able to tell what type we hold in hand (String for example), but we will not be able to deduce what union of types it comes from (Null¹|String or maybe Null²|Null¹|String).

Could it be this cookie is absolutely for free? Unfortunately — no. Union types does not work well with generic types (at least if you want to keep static control over types), but since we have here very specific case of union we can enrich type info with option counter. Whenever there is Option<T> type used we have to take option counter from type T and increment it.

This leaves me a syntax to think about and supporting three valued logic to consider — this could add an interesting twist to the language.

What is easy — generics with nullable types not for sure

For a week I have been removing features from Skila (so far just two) and postponing some nice things for later implementation — I still want Skila to look at least nicely. Ironically that easy subset is not that easy to grasp, currently I hit hard on nullable types. So far I could see three camps:

• C# — implicit null-check with null values (there are no referential non-nullable types though),
• Kotlin camp — explicit null-check with null values,
• Swift camp — as Kotlin but with wrapper class “Option” instead of null.

Take my words with grain of salt because I don’t know Kotlin or Swift.

Using “null” is not the same as having “Option”, the second can be wrapped as many times as you like when in case of the former you cannot wrap “null” into “null” — it is a value, not a wrapper.

On the other hand all checks designed for nullable type can be applied to non-nullable type, this is not true with optional types. For example one can blindly execute “x != null” check in Kotlin, while the analogous code in Swift requires “x” to be really an “Option”.

Worse, both Kotlin and Swift approaches bring problems to Skila, for example — how do you initialize objects of generic type? If I choose Kotlin solution how does inheritance tree look like for nullable types? And even if I choose Kotlin approach indeed I know that “Option” type will have to be added as well, so maybe add it right from the beginning… and so on and on.

Just to increase my confusion I found out some inconsistency in Kotlin:

open class Foo {
  fun foo() : Int {
    return 0;
  }
}

fun test<T : Foo?>(x : T) : Int {
  return x.foo() // (*)
}

fun main(args : Array<String>) {
  println(test<Foo?>(null))
} 

Compiler should demand “x!!.foo()” in line marked with “(*)”, but the code above compiles without any errors. And it worries me for Skila — those two ideas (generics with nullable/optional types) are not easy and it is hard to foresee (at least for me) all the problems with writing generic classes in practice.

If you are curious you can play online with both languages — playground for Swift and for Kotlin.

Nulls are OK

Judging by the hot disputes about null it should be some kind of the problem, but I have to admit I don’t see any. A lot of people complain about getting NullPointerException (and alike) as a bad thing — for me it is great. It is the mechanism that stops my program and tells me “listen, the flow of your program is incorrect, I prevented bad things to happen, sit down and fix it”. I am grateful for that because I didn’t add any extra line to my program and yet the runtime keeps me safe. null values are very useful if you start to treat them as normal citizens among your data, same way as you treat zeros when doing computations.

a = b + c;

Are you sweating because “b” could be zero?

Some languages try to cover null “problem” by introducing wrapper class for reference/pointer types (for example Scala with its Option ¹). However what kind of progress here is made with actually renaming null to None? I don’t see any ².

Since C# and extension methods I see things differently — to such extent that I started writing extensions instead of regular methods just to allow null on the caller side (to this day I remember pride of C++ which guarantees “this” cannot be null).

In many cases you have to deal manually with null, period, but in many, especially when the job is related to collections, null comes naturally. Consider such task — having HTML node extract first “<p>” node from its children, and then extract all “<span>” nodes from its children. If you use XPath query you have one sequence of selectors. How come in classic programming you have to add so many guards against null? Why not solve this without fear of null?

var spans = ((node??emptyNode)
              .Children
              .Where(it => it.Name=="p")
              .FirstOrDefault()??emptyNode)
                .Children
                .Where(it => it.Name=="span");

The only problem, or rather annoyance, is C# — not null.

Kotlin addresses the issue directly but misses the point — instead of introducing coalesce operator it allows subsequent call/access to fail (Kotlin “returns” null):

var children = node?.Children;

If “node” is null, the variable ”children” will be null as well (Fantom and Groovy also took this approach). I am not saying coalesce is superior in all cases, but adding a feature which moves you from null domain into the same null domain does not look like a solution to me.

Skila will use selective Null Object pattern ³ with short coalesce operator. For every type you can define a constant null object for coalescing. And then…

var spans = node->Children
            .Where(it => it.Name=="p")
            .FirstOrDefault()->Children
            .Where(it => it.Name=="span");

“spans” won’t be null! Arrow operator (“->”) is hidden coalesce call. Because Skila is statically typed checked, it verifies at compile time whether you can use the power of coalescing (it checks if the null object is defined). And it does the right thing — it moves you from null domain into not-null domain.

This is probably not a complete blueprint — practice will be the best judge — but even upon completion it won’t be a magical wand. If you work on types that are collections, adding coalesce is like adding a grease into the gears, in other cases you should rely on old “if”, and for the rest — NPE comes to the rescue.

---

¹ I say ”yes” to Option, but not instead of null pointers, but as addition to them.

² Why Scala’s “Option” and Haskell’s “Maybe” types won’t save you from null (good read, but the example with Dictionary is weak).

³ There might a be a little confusion what the true “Null Object” pattern is, so let’s say Kotlin, Fantom, Groovy use thin null object (no body, everything falls into nulls), while Skila will use fat objects, with real body defined by user. This will allow to transform null collections into empty ones.