I am glad to see that this is optional. We need languages that mix compile/parse time (static) and run time (dynamic) types, to mix what fits best to each situation.
Of course type checking can catch some problems. The more extremist elements of the static types proponents seldom seem to ask the cost of adding types, and making some otherwise generic/overloaded operations harder to implement (e.g. - excess clutter in some "wordy" type systems, particularly in the C lineage where declared identifier names follow the other verbiage). We need more tools that allow us to easily nail down fixed interfaces to avoid errors, but still allow flexible / dynamic / metaprogramming type code when needed, without having to jump through hoops in either case.
Don't be a static typing extremist! (or throw the baby out with the bath-water, either, I guess)
I dunno, I find static types to be a core design tool that are highly beneficial from the earliest stage of a project. They help you probe a problem space as you develop, and offload a great deal of cognitive burden to your tools. I actually find myself working a great deal slower on new dynamically typed projects because I have to juggle a great deal more context in my head, and am encumbered by larger test suites that are more difficult to change.
Note that I am referring to type systems with succinct syntax, type inference, structural typing and ADTs.
If you can't keep the "types" of your design in your head, your design might be broken. I have a real bad short-term memory and can probably juggle less things in my head than most programmers. Yet with good taste for design / decomposition that's not much of a problem. Most of my code now is structured as tables, which greatly simplifies many transformations. There is no pressing need for typing tables, and anyway most of these type systems really suck at "tables transformation algebra".
Here is a little list of drawbacks of static type systems with features like you describe (parametric types, ADTs...):
Type dependency hell (banana, monkey, jungle, etc), having to fight/convince the type inferencer to accept any but the simplest things, having to type many things twice.
Looking at Haskell, "almost everything is inferred" is a lie in my experience. You are encouraged to write code using higher-order and highly parametric functions. If you do, the error messages you'll get if you don't annotate are plain sick. Another problem is that the type system provides no convenient escape hatches, while it's not powerful enough for typical programs. Many "successful" example designs I've seen are either _really_ simplistic (to the point where it's obvious that the better design wasn't chosen simply because the type system makes it really inconvenient), or have some _really_ terrible internals hacking in it (which might be the largest part, be impossible to understand by most developers, and break with the next compiler version).
Looking at C++, a practical example is vector. It's a useful tool (automatic memory management), and very likely to be all over in your function signatures. This is very bad in terms of modularity / reusability, because your functions are not defined using reasonably general types (pointers to arrays).
Many of these very real pains just aren't there with Python code, which has better modularity at the cost of being a little sloppier.
These days I look at static types mostly as tools for generating fast machine code (I doubt that much more than C types is needed here) and as tools for enabling nice syntax (OOP protocols like in Python can do that too, but they are very dirty). These things however are only needed at the very optimized parts of your average project.
So I think a good way in many cases is to go the Python route, and check types dynamically at strategic points to assert value integrity and enable transition to highly optimized (with static types) routines.
Granted it's difficult to remember the type of the casual e-mail address or of a list of integers. In Haskell, where there are a thousand ways of encoding the most trivial things, that is.
In my own practice, most transformations don't involve relations over more than 5 variables with mostly obvious types. Yes, this is hard to use:
>Granted it's difficult to remember the type of the casual e-mail address or of a list of integers.
It would be a toy project that only (or mainly) has those as all the types you need to remember -- so that just proves my point.
Also the 'type' of an email address is an unsolved problem -- if we're talking type checking. It's not a string, and it's not solvable with regex either. So bad example all around.
You cannot do any kind of Domain Driven Design with dynamic typing.
In F# you can have a type email address and it will be impossible to pass an unverified string there.
It has to be a valid email address, and you will know it at compile time.
Good luck to try to achieve the same effect in python without writing tons and tons of useless tests just to check the type.
And if you can keep all the type of your system in your head I don't think you ever worked in a real system.
You could make a property in Python that validates it's given value and throws a ValueError if it's not an email. 6 lines of code max, with maybe two tests.
I have gone that route in Haskell and it was a mess. In the end you need to extract the string again (basically everytime you do something meaningful with it). All these pattern matches kill code readability/writeability.
Also, in Haskell one needs to encapsulate the real constructor in its own module to make sure the data is only constructed through a sanitizing constructor. That's quite some overhead (and makes pattern matching impossible).
What might be useful instead is first class "tags". In Perl6, you can subset types to values that match a condition (for example, strings that match an email regex (cough)). They are still strings so can be used unchanged with other functionality that does not need the tag. It seemed quite usable to me (applications are limited, though). However the creator said it's still expensive because they haven't figured out / implemented how to transfer invariants, so they are checked at each function boundary.
In Python, you can use duck typing: subclass str, it's a dirty hack but works very nicely.
The real world solution to the whole mess seems to me to be just to design a good dataflow into the system, so it's clear where you have a valid email address. Typically it's very easy: Just assert the invariant at the boundary to the external domain (for example, HTML form parsing). Now whenever there is a variable "email" in your program you "know" that it is a valid email address.
I’ve gone that route in Haskell and it works great. Yes, you need to extract the string again. Not an issue unless you’re golfing all your code. You also don’t need to hide the real constructor if it’s a type your module should be working with.
Subclassing str seems like the most horrible way ever to represent an e-mail address. I would definitely write a wrapper instead – exactly like Haskell.
In python nothing can guarantee at compile time that you are not passing a username variable in a method that accepts an email address.
You'll just notice at runtime when you realise that you have to rollback your last deployment because new users are unable to receive the activation email that is being sent to their username instead of their email.
If you’re talking about runtime type checks rather than mypy, well, they’re just verbose static types with less safety. (May as well just embrace the duck typing at that point by accessing email.local_part and letting the AttributeError propagate.)
You’re making out having to explicitly specify a type when type inference doesn’t do a satisfactory job as some kind of extremely painful thing, but it’s not. You just define the type of your object, which is something you had to know anyways; now it’s written down and people reading don’t have to guess at or remember what it is.
Annotating even the last bit of code that uses type class methods until the error message becomes clear is very cumbersome. For example, I can't say (traverse :: The List instance), but have to dig out the whole annoying signature, or go look for a more strategic value where the same information can be put in fewer characters. For another thing, it often duplicates the amount of lines needed (or more if you like to put in whitespace before types). It's definitely very annoying.
I'd be interested to hear more about Haskell projects where the type system causes bad designs. Maybe you could write a blog post about it or something?
I will think about it, but would probably not be comfortable with it.
For now a few words that I hope I won't regret later (apart from the things I mentioned): The bulk of invariants / assertions in many real-life programs can't be expressed with mainstream type systems at all. They are too specific or only come to life dynamically. Things like: This array of integers is sorted ascendingly, and that index is a valid index into this other array which points to a number between 48 and 58. Or this table's column over there contains only values that are also present in that other table that just happens to be present now (but must be expunged from working memory after the next subtask). Checking if an integer fits in 8 bits and then converting to a byte is no safer in Haskell than in any other language.
So much to put the usefulness of mainstream types for static correctness in relation.
Now much of the tinkering that is going on in Haskell projects seems to be shoehorning yet one more real-world invariant in another crazy type. Maybe the next brilliant idea works for one more toy case in the end. Great, but the cost/benefit ratio is just bad.
I think the most important thing in software architecture is modularity, because the most pressing need in software development is the need for change. If my project is very modular, it's sufficient to have clear entry points to each module with very simple data types in the signature. The internals can be a dirty hack and rewritten later, or the module thrown away when the idea turns out bad. Interfaces on the other hand can't be changed as easily since they affect much more code. So a good approach is making interfaces that accept tables, augmented by a textual description of invariants so the user easily "gets" the meaning of the data and can reshape/preprocess the data as needed. This enables isolated rewrites of modules. It also enables creative use of data in unforeseen ways. Clever types on the other hand have a very strong influence not only on the design of module interfaces but also on the dependent modules' implementations, because 1) it's more cumbersome to reshape complex structures in a way suitable for clean implementation, and 2) because type systems can't usually transfer invariants to reshaped representations (and after all the goal was to let the compiler handle the invariants, so reshaping is a no-go).
In short, complex types _kill_ modularity because they have far too much influence on data layout.
For real world programs, what Haskell is great at is statically finding trivial interface usage errors like, don't call this with a NULL pointer, or this parameter must be a functor instance (because the actual data might be too ugly to admit). Turns out that that's all nice and well, but the machinery is a huge cognitive and economic overhead, while running some python client code of a sane API once typically brings such usage errors to light just as well (profane things. TypeError raised. Seems it tried to iterate over an int, so you must have gotten the argument order wrong etc).
>For now a few words that I hope I won't regret later (apart from the things I mentioned): The bulk of invariants / assertions in many real-life programs can't be expressed with mainstream type systems at all.
Which is beside the point, since partial coverage of invariants is still very useful.
Having a small subset of invariants checked by the type system for sure is beneficial, however I was saying that the cost/benefit ratio is just not good enough (IMO of course: as a non Haskell expert, who is not a genius and has only spent 2-3 years learning it...)
> This array of integers is sorted ascendingly, and that index is a valid index into this other array which points to a number between 48 and 58
These are difficult to encode in a way that lets the compiler check the code that sorts or calculates an index.
They are trivial to encode in a way that simply tags the values. I still need to be concerned with making sure my sorting is correct, but that's a very isolated concern. I still find it very useful to know whether I've called a function on the array that depends on it being sorted, and that doesn't require any heavy lifting at all.
I haven't had good experiences with tagging. There are just too many "tags" (often one per line or group of lines).
They also have disadvantages in that they separate data which is only different by some aspects, but not most others. (For a simple example, the printing function is not interested in a "safe HTML" Tag. The effort needed for constant tagging and untagging (or using "Stringy" classes, or the cognitive overhead you need to pay if you found a clever way to remove the syntactical burden) is too big except in very few cases.
> There are just too many "tags" (often one per line or group of lines).
Where the burden outweighs the benefit, you always have the freedom of typing things too loosely - to be sure, without something like refinement types, there is a granularity you can easily try for that's just not worthwhile.
But that's a pretty different thing than "can't be expressed with mainstream type systems at all." For me, the questions are 1) how far does this type move information in its use through the program, and 2) what part of these are likely to change in ways such that I'll want to be notified about this piece of code here?
> Looking at C++, a practical example is vector. It's a useful tool (automatic memory management), and very likely to be all over in your function signatures. This is very bad in terms of modularity / reusability, because your functions are not defined using reasonably general types (pointers to arrays).
That's why you define your functions as taking generic Iterators instead, which generalize pointers to arrays. Or, in more modern style, generic containers.
You can wrap that in a type erasure layer (lookup any_iterator) if you want separate compilation.
> That's why you define your functions as taking generic Iterators instead, which generalize pointers to arrays. Or, in more modern style, generic containers.
Or, in yet more modern style as slices. Or, ...
The only thing I miss from pointers + indices is optional bounds checking. Otherwise, it's the perfect tool for productivity (because readability, writeability, portability, modularity).
> Most of my code now is structured as tables, which greatly simplifies many transformations. There is no pressing need for typing tables, and anyway most of these type systems really suck at "tables transformation algebra".
What do you mean by "tables"? I think of a database table of rows/columns, or a matrix.
Sorry to vent on you, specifically, but many people seem to have missed the point where I said I wanted them.
I just don't want to be forced into a jail cell 100% of the time. I want an escape hatch to do the needful once in a while to do something that the language designer never planned for.
A sufficiently good static type system won't make generic operations harder to implement. It can even make them easier to implement in terms of guiding you towards good generalization techniques.
You can also do full Python-style "duck typing" statically. It's called structural typing, with native support from languages like typescript, purescript, etc. It's used a lot in web dev because it works well with Javascript's object model.
> A sufficiently good static type system won't make generic operations harder to implement.
This is true for the JVM definition of 'generics', but it doesn't mean that arbitrary operations on a collection of types can be easily expressed with a given (static) type system. Run time type information + a dynamic type can ameliorate some of this, but that's shoehorning optional types into a static type system, and I don't think that's quite what you meant.
Think about a generic operation to diff two data structures: you need access to the members of the struct in a generic way. I don't think this is easily solved with static types.
Can you give a specific example where you think you need dynamic types? The diff operation you described sounds like it could be made generic pretty easily, but I may be misunderstanding the goal.
> Can you give a specific example where you think you need dynamic types? The diff operation you described sounds like it could be made generic pretty easily.
How do you figure? This would require macros or runtime type introspection.
A specific example might be "serialize this json type without having to annotate the fields with how to serialize them".
True, but "batshit conservatives" won't do that, or will mess themselves if you do that, rather than making some kind of "Object-Whatever-Mapping" model and annotations.
But that's what I would do (at least in Java) for this kind of transient transfer object, if left to my own devices.
"deriving" is compile time; "Generic" is runtime inspection of the shape of a value (in terms of constructor applications). Something (at least vaguely) similar to Generic exists in many statically typed languages - including (IIRC, IIUC) Java and Go.
> Something (at least vaguely) similar to Generic exists in many statically typed languages - including (IIRC, IIUC) Java and Go.
Sure—but it's in spite of static typing, not because of it. You certainly give up any guarantees of avoiding runtime errors by detecting typing issues at compile time.
Sure, it's orthogonal to static typing (I wouldn't quite say "in spite of"). I wasn't trying to weigh in on the general topic, just to clarify what was happening where.
Yeah, Java has kind of "choked the ecosystem" for the last decade. That monster needs to retire before the industry can move forward (the language, at least, the JVM seems to work well).
Java generics seem easy enough to instantiate and use, but the mess when you implement one really discourages people from trying, usually. And if you do, the result is often a poster child for FUGLY code. Try declaring a java 8 lambda parameter that works with collections some time, and using that in a few places - serious COBOL-fingers and eye-burn time. Even a Pascal style "type" or C style "typedef" mechanism would have greatly reduced clutter when trying to create code that implements (java) generic operations.
OCaml-style structural typing cannot handle deciding at runtime what fields an object has. Arguably it's best to know this statically, but dynamic language users make extensive use of the ability to dynamically add fields to an object.
Some time ago on HN there was Facebook exploit posted where it was possible to track your friends. All this was possible mainly because lacking real types the only stable interface becomes string and everything becomes stringly typed.
Static types are not only strings and numbers and static types are about something more than typed variables. Too many times have I run into situation when I call some API in dynamic language and have absolutely no idea what can I do with the thing being returned. In dynamic language I have to either read the docs (hoping they are good) or introspect in a REPL, IDE won't help. And In Python world, the wrapper library method I'm calling forwards kwargs to some underlying library and I have absolutely no idea what can I pass without getting familiar with the underlying library. Dynamically typed abstractions are very prone to leaking.
> We need languages that mix compile/parse time (static) and run time (dynamic) types, to mix what fits best to each situation.
C# has the dynamic keyword (not to be confused with type inference via the var keyword), which basically allows to break out to dynamic typing. I find this to be significantly saner than the other way around.
I think there's a false distinction between static/dynamic types. As if all static type checkers offered the same benefit.
IMO writing a type annotation, or knowing whether a type is an int or a string, is the of minimal help in pushing errors into the type system. A good type system you don't even have to write the type of things, it's inferred.
Better 'statically typed' languages allow you to push more errors into the type system and give you tools for creating stronger abstractions.
Somebody, somewhere, is waiting until runtime to check for a property, and they will burn in Hell for their sins!
Even if they don't do it very often, even if the IDE infers properties most of the time anyway, even if there's less code to have to be refactored, and most of the refactoring that could be automated is still automated.
I would rather use types most of the time for clarity and to help the compiler / editor help me, just not always.
Of course type checking can catch some problems. The more extremist elements of the static types proponents seldom seem to ask the cost of adding types, and making some otherwise generic/overloaded operations harder to implement (e.g. - excess clutter in some "wordy" type systems, particularly in the C lineage where declared identifier names follow the other verbiage). We need more tools that allow us to easily nail down fixed interfaces to avoid errors, but still allow flexible / dynamic / metaprogramming type code when needed, without having to jump through hoops in either case.
Don't be a static typing extremist! (or throw the baby out with the bath-water, either, I guess)