That's unfortunate. RunAbove was very nice and gave us one last year for Haskell.org (so we could port to PPC64le), and our instance is still running well (176 cores/48GB of RAM or so), but I can confirm I can't spin up any new instances.
That's a shame, they're excellent machines... I guess we'll have to find another sponsor soon.
Haskell does not eliminate side effects. It allows you to control them.
You do realize that system you described - Facebook's Sigma - is a real-time, massively concurrent online spam-fighting system, that automatically and implicitly makes your code concurrent for all operations, can optimize it, and has automated caching/batching request mechanisms in place to talk to external systems (Graph databases, SQL, cache servers) in an optimal way while reducing things like round tripping, with naive code? (e.g. it can automagically parallelize and optimize the "N+1 Query Problem" into an efficient query that minimizes duplicated requests and round trips, with no intervention, and the single 'query' can operate over multiple data sources like mentioned previously.) In other words: it does side effects all the time, and makes them manageable in ways you could only dream of in other languages. Did I mention all of that was done in a library? :)
I mean, we definitely have a sales problem, don't get me wrong. We could spit these stories better or clean up the presentation. I can list off a dozen things Haskell is poor at, or probably an infinite number given the time. But you sort of chose literally the worst example in the world to make your point.
I would suggest you actually spend some time with the language in anger. It is not easy. But I've been writing 'real world' software in Haskell for years, and it's no more difficult or 'magical' than any other part of being a software developer. It's just more rewarding because my code works far more often. And working code seems to be something that's rare in the software world these days ;)
That's a great example of a challenging system, analogous to things I've worked on in the past, so I can feel the complexity, and your description also helps me sense the way Haskell is well suited to the task.
So that's cool. I'm glad to hear it. The best example found to date.
And you gotta admit, crdoconnor has a good point that the impact on the world at large may increase as lessons learned in Haskell spill over into the Lesser Languages.
I still hope that some of these super smart guys would take a short break from the language wars. And briefly at least, consider working on things with a more tangible outcome.
You misunderstood me. I wasn't saying that it eliminated side effects. I was saying that it brought no great advances with respect to side effects, and most people write code that mostly only does side effects. This is why OO had such a great impact on the business world and functional programming didn't.
It's actually pretty rare for businesses to write highly complex mostly functional code (a spam filter is one exception), and that is where Haskell really shines.
>I would suggest you actually spend some time with the language in anger. It is not easy. But I've been writing 'real world' software in Haskell for years, and it's no more difficult or 'magical' than any other part of being a software developer. It's just more rewarding because my code works far more often. And working code seems to be something that's rare in the software world these days
I've tried it and I've seen its advantages and I'm content to remain with python for the time being. Python's type system isn't as good and that does sometimes cause bugs I wouldn't get in Haskell which I fully understand, but I'd consider the difference incremental. It's not any kind of great leap forward and I think my code would only be slightly less buggy in equivalent Haskell.
On the other hand, Python has a wealth of packages and an ecosystem that Haskell simply doesn't even come close to matching.
Stable, working code is rare, I'll grant you that, and weaker type systems definitely make programming more of a balancing act, but there's a trade off to be made between stronger type systems and not having to rewrite a ton of working code which already exists.
> I was saying that it brought no great advances with respect to side effects,
Really? I work with a bunch of languages that does not have controlled side effects, and controlled side effects is the single biggest thing I miss from Haskell. The uncontrolled side effects I swear over daily, if not hourly.
The fact that the order in which I call functions can – undocumentedly – decide whether my program works or crashes is a ridiculous concept. The fact that if I share the wrong piece of data with another part of the application my program starts behaving erratically from having invalid data is odd.
Sharing data should not feel unsafe. Changing the order of method calls should not be a threat.
As long as a function gets the data it wants through explicit parameters, it should work. It shouldn't matter at which point in time I call it, because managing time is really difficult. Managing data is easy.
The controlled side effects in Haskell give you a way to encode these things to give you guarantees and self-documenting code. It gives you opportunities to deal with these problems in sane ways.
Refactoring Haskell code is mostly a matter of symbolic manipulation. A huge chunk of the process doesn't even require me to think about what I'm doing, because I have much more freedom when I don't have to worry about side effects myself. This is good, because I'm bad at thinking and I would like to do it as little as possible. Every single bug that have appeared in my problems have been because I'm bad at thinking.
> I was saying that it brought no great advances with respect to side effects,
Sure it does. Well, side effects are boring. It's how you actually do the dance that matters. Here's a few:
- Haskell has a best-in-class concurrency story, with a far greater breadth and depth of techniques and libraries available in most languages (and some, like STM, are realistically impossible otherwise). These include many manners of graph, dataflow, (implicitly auto) parallel and concurrent programming.
- We have a nice blend of epoll based event loops with a multicore runtime, all compiled to native code. This means the traditional 'one thread per client model' is as efficient as any epoll/event loop solution. In practice, it is so cheap because Haskell threads are so cheap that it completely changes how you use concurrency, as it is almost free. It is cheap and easy to add, and the aforementioned toolbox makes it easy to do things - e.g. the `MVar` acts as a one-place blocking queue, which you can use as a lock or shared data structure, as you see fit, between threads.
- Haskell's type system has many features most other 'side effectful languages' don't offer, and even ones that are completely impossible even in things like dynamic languages.
For example, it's possible to use the type system to do things like ensure file handles are tracked statically so they cannot escape a certain scope (a form of 'region management', tracked at compile time). This is actually not even that difficult, honestly.
- Similarly, things like type classes offer a powerful set of abstractions for working with 'effectful code', and allows us to do things like easily separate interfaces from implementation. I can specify a 'Redis' type class and then a concrete implementation of this 'Redis' class, which may or may not be a fake or real. Then I can write functions parameterized only over this type class, and it works with both, and I choose which to use. If I push this further with more classes, individual functions may have a 'Redis' context and a 'MySQL' context to do multiple things; or it may only have a 'Redis' context, and no MySQL. This can be used to enforce separation.
- Lots of use of the type system to enforce all kinds of special cases and invariants. For example, it is entirely possible to use the type system to avoid things like SQL or XSS injection attacks, as done in Yesod. For the extreme version of this, the Ur/Web language takes it to the ultimate conclusion.
- Similarly, it's very easy to encode business logic in Haskell types. You can ensure things like you never mix up a "CustomerID" value with a "ClientID" value, which are really both Ints, but you can make the compiler yell when you mix them up or get something wrong.
- Haskell is a lazy language, meaning it is very easy to refactor, even I/O code. That is because you can pull any piece of code out into a name. Consider this program:
if thing then error "bad things" else 10*2
I can change this to:
let x = error "bad things" if thing then x else 10*2
You cannot do this in a non lazy language. Well, you can, but it gets miserable pretty quick. Furthermore this applies to any subexpression; suppose I have a program:
h <- openThing "foo"
doStuff h
... some more stuff ...
undoStuff h
closeThing h
I can change this to:
let begin f =
h <- openThing f
doStuff h
let end h =
undoStuff h
closeThing h
h <- begin "foo"
... some more stuff ...
end h
Again, this is not valid if your language is strict. But these patterns are very obvious and common when you can freely rearrange any expression. Then you can see the final abstraction easily:
let withThing x f =
h <- openThing f
doStuff h
f
undoStuff h
closeThing h
let stuff = ... do some stuff ...
withThing "foo" stuff
This particular point requires a lot of experience to truly appreciate in practice IMO. But in practice it means you can freely move code anywhere at no cost, even any I/O code, so you can very easily abstract over it like above. In non-lazy languages, this becomes significantly more tedious.
> It's actually pretty rare for businesses to write highly complex mostly functional code (a spam filter is one exception), and that is where Haskell really shines.
The term 'mostly functional code' is meaningless, because we have not even defined what it means, so it's not really useful for me to try and convince you of anything :)
Again, you do realize the 'spam filter' that Facebook produced isn't just some weekend Bayesian spam filter, right? Actual software at the scale is quite complex. I'd assume it handles at minimum tens to hundreds of millions of requests a day and integrates with millions of lines of internal code. I'm probably lowballing that, even. It's a serious piece of software.
> I've tried it and I've seen its advantages and I'm content to remain with python for the time being. Python's type system isn't as good and that does sometimes cause bugs I wouldn't get in Haskell which I fully understand, but I'd consider the difference incremental. It's not any kind of great leap forward and I think my code would only be slightly less buggy in equivalent Haskell.
That's fine. But based on what I've seen (to be completely honest, not as an insult, you do not give the impression of someone who has spent extensive time with it[1]), I'd say you're greatly underestimating exactly what it is capable of, as well as precisely what capabilities it offers in the first place.
[1] The fact it should take you so long to understand it is a problem in its own way, and we should certainly keep seeking to narrow the gap between "regular ordinary programmer person" and "someone with a weird disposition for Haskell"-slash-"Someone who has way too much time"-slash-"Someone who is super persistent". Totally a problem.
> On the other hand, Python has a wealth of packages and an ecosystem that Haskell simply doesn't even come close to matching.
This is certainly a real complaint for a lot of domains. It's one of the many things I could complain about. In other domains, Haskell is excellent and has quite a lot of good options. It seems to be particularly popular for web developers these days. It's definitely still pretty terrible if you want like, QT bindings on Windows.
The concurrency stuff sounds nice. Apart from that I'm underwhelmed.
I can handle epoll based code without problems anyway.
The whole "file handles" can't escape a certain scope sounds like what python's with statement does.
The whole it's amazing that you can "encode business logic in types" thing I really don't get. That's basically the selling point of object oriented code.
Similarly, SQL injection is a problem that largely plagues dumb PHP code and is close to irrelevant if you use a half-way decent ORM in any language.
>The term 'mostly functional code' is meaningless
Any code that is mostly made up of functional programming, but still has some state handling code. I think it's pretty clear.
>Actual software at the scale is quite complex.
Gee maybe that's why I said it was complex.
>That's fine. But based on what I've seen (to be completely honest, not as an insult, you do not give the impression of someone who has spent extensive time with it[1]), I'd say you're greatly underestimating exactly what it is capable of, as well as precisely what capabilities it offers in the first place.
No, I haven't used it extensively. I've played with it.
I honestly think if it the benefits you tout were as great as you seem to think they are then there would be a far greater wealth of practical software written in it.
I know of one website (a local fashion retailer) and Facebook's spam filter. That's about it.
> The whole "file handles" can't escape a certain scope sounds like what python's with statement does.
It is incredibly easy to return a file handle out of a with statement. There is nothing in place other than programmer discipline preventing you from doing so, and your program blows up in... interesting ways.
> The whole it's amazing that you can "encode business logic in types" thing I really don't get. That's basically the selling point of object oriented code.
With OOP languages those are mostly runtime errors rather than compile time errors. Encoding logic into the type system means that many application logic errors will be caught at compile time before your application goes into production.
> I honestly think if it the benefits you tout were as great as you seem to think they are then there would be a far greater wealth of practical software written in it.
I honestly think if PHP were as bad as it seems there would be far less practical software written in it. Popularity doesn't imply quality just like lack of popularity doesn't imply lack of quality.
Two very clear problems are that it introduces more complexity up front for beginners (because now to understand `length` you need to understand the concept of type classes) and the change had the possibility to break some extant code, due to it changing some type inference properties.
In practice, I think #2 was a fairly uncommon case, and the fixes were trivial and backwards compatible, so the cost was deemed pretty acceptable. #1 is a point of debate, because 'beginner' needs context (perhaps experienced programmers that are Haskell beginners would get over it quickly, but non-programmers would be substantially more confused, etc). Unfortunately I don't think we have a lot of truly empirical evidence on #1.
My empirical evidence is against #1: up-front productive learning beats permanent uncertainty.
- I can fold a list, but what about other types? Which of the similarly or identically named functions dispersed among a bewildering number of library modules is the right one? Are they any different?
- I can fold a Foldable? What is it? Let's find a tutorial about Foldable. Ooh, nice! That takes care of folding anything that can be folded! If I'm not folding a list I only need to find where the relevant Foldable instance is (very easy) or to write one myself (reasonably easy).
IIRC, Signal/RedPhone takes a (truncated) SHA-512 hash of your phone number/email after verification of your account and sends that to the server, and does the same for your contacts. If there are matching keys, it does the exchange for you quickly and easily, and this is basically all the server does for 'user management' I think. So your friend can have their phone/email intercepted, but the central server isn't going to reveal much data or anything at least.
Second, you can verify fingerprints manually if you're in person. The biggest draw is there is no distinction between a user who you've simply exchanged keys with vs a user who's fingerprint you've verified. This is something the Threema messenger gets right. Trying to explain capabilities of the attacker and how you could be MITM'd to some random user is totally pointless and will just scare them, it's detrimental to adoption. It's far better to have a visual indication of how much relative 'trust' you have with your individual contacts, rather than write a novel implying there could be G-Men on the other side of the line.
Finally, for phone calling capability, the loop can be 'closed' through a second level of verification, because RedPhone/Signal give you a (matching) set of random words upon connection establishment, and each party says the secure words to each other before anything else. This was an idea taken from SilentCircle, I believe.
The idea here is that it is easy for a human to verify they're talking to someone they know simply if they hear their voice verify the words they're seeing, while it's probably going to be difficult for an attacker to imitate arbitrary voices on demand in real time to 'spoof' a human.
> but the central server isn't going to reveal much data or anything at least.
As moxie acknowledged himself [1], the space of "all phone numbers" is so small that bruteforcing suddenly becomes feasible. AFAIK they're still working on it but I'm not up-to-date on what's been done since.
Nix is what Guix is based on. You can think of Guix as the 'GNU Version' of Nix/NixOS, meaning it provides free software packages, and IIRC it uses Guile Scheme, rather than Nix, to describe all the packages and OS configuration.
It's a toss up as to whatever you like. I think Nix has a bigger community, more packages, etc. But Guix, strictly speaking I think, is a 'superset' because changes from Nix often flow into Guix (and Guix can use Nix packages directly), as Guix is built directly on the same source-code as Nix - but not the other way around. There are some other differences, like the fact Guix uses GNU dmd while NixOS uses systemd, etc etc.
It's all just personal preference I think. I use NixOS because it's reliable and has a decently sized community, and a lot of packages. It also has really, really good Haskell support, and being a Haskell developer, that's a pretty big plus to me. Having non-free packages isn't so much of a stickler for me. I think the Guix people are doing good work, though, and a distribution for free software and the GNU project is very important - so I wish them the best even if I'm farther away.
You'd have to try both of them and make a decision for yourself IMO. But I warn you: the rabbit hole is deep, and will require learning. And when you come out - you may be immensely displeased with the current state of affairs. :)
I'm giving both a try. Nix is more polished, has many more packages, and is better documented.
I prefer Guix choice of using a real programming language (Scheme!) instead of a DSL, but I really like Nix anyway.
Something that annoys me sometimes is that Nix has a few really bloated packages, compiled with all dependencies on. For example, if I try installing mutt I eventually get python as a dependency. This is a bit ugly. I'm aware it's easy to change this, but I'd still love to get thinner binaries from hydra. Otherwise, a really neat piece of software.
Guix tends to be a bit more like Slackware or Arch. Very vanilla things. I would love if Nix went a bit in that direction too with regards to packaging policies. It's more secure and nicer to humble devices, like cheap Chromebooks.
> Well, when the previous announcement was made, I immediately Amazon'd some C books, which I plan to devour in my free time. At which point I'll be learning Rust, and reimplementing LuaJIT in Rust, and hopefully convince Mozilla to host the git, such that it will be protected from FOSS corruption.
The point isn't the LOC. If you've never even touched C, you're not just going to have to learn that, you're going to have to learn how to write an optimizing compiler (because frankly if you've never touched C I'm skeptical you have any experience in this). And not just that: you're going to have to learn how to write the world's most optimizing trace-based JIT compiler for a dynamic programming language.
Mike spent 10 years designing LuaJIT and it is in a league of its own, not paralleled by anything else. Do not expect this inane project of yours to be solved by looking at 25,000 lines of C code. Especially if, point of fact, you do not even know C. Expect it to be 'solved' after a decade of research and hard work at minimum.
I'm not sure what paranoid reality you live in where you think this is feasible, or even desireable given your original post (frankly even though a port isn't needed Rust would be an awful choice for a 'port' due to the fact it's simply not got as good availability, the compilers and tools are less mature), but when I said see you in 15 years, it wasn't a joke - it was a conservative estimate.
I think the horsepower on those machines shouldn't be underestimated, because they are not entirely as equivalent as you think... I was thoroughly surprised when an unoptimized (but correct!) ChaCha20/8 implementation I wrote on a 3.0GHz POWER8 little-endian machine was about as fast as the latest 3.5gHz Xeons @ AES-256 with AESNI (about 1.3cpb vs 1.0cpb IIRC, but the latter has a dedicated hardware unit for it!) On that same Xeon, the ChaCha20 code only hit somewhere around 5cpb - that's software vs silicon!
It also has 170 cores and was actually a QEMU instance (w/ hardware virtualization extensions) vs raw dedicated metal. If you're doing any kind of numerical or analytic workloads (even databases), I wouldn't throw them aside so quickly. You can even get CUDA for them these days, and certain physical addons like CAPI allow you to map and coherently share physical CPU address space with FPGAs or GPUs. If I could get those things in a reasonable workstation configuration, I'd probably go for it tbh.
(I'd be more than willing to repeat this and post some more accurate numbers if anyone cares. I also need to get around to benchmarking AESNI vs that POWER8 machines _actual_ dedicated AES unit. The benchmark above was only flexing its vector/integer unit capabilities. ;)
If you're getting a 4x difference in IPC using a crypto microbenchmark from compiled C code (i.e. it doesn't sound like you're bandwidth or I/O limited), there has to be something else at work. POWER8 is a nice core, but it's not that wide. Maybe the compiler was recognizing your operations and replacing them with AES primitives?
Yes, but at this kind of multiplier only in the case where the entire test is 100% cache-resident on one CPU and spilling on the other. Crypto stuff tends to have small working sets, so my intuition is that it's got to be something else.
One thing that makes it 'worse than nothing' is that it contributes to amplification attacks due to large response packets: http://dnscurve.org/amplification.html
You'll probably be fine either way, IMO - OCaml has a lot going for it, it's fast and incredibly quick to compile, and has a pretty straightforward execution model. Haskell is not quite the same (different evaluation strategy, different ways of organizing your programs), so there's a bit of associated overhead there, and for a beginner it can appear daunting.
The most important part of these languages is really how they try to enhance your abilities to write modular programs. You might be surprised to find out languages from the 1970s and 1980s have better abstractions (e.g. functors/modules) than some designed today. :)
But fundamentally I think some of it will come down to personal choice and aesthetics at some level; e.g. I really like Haskell's syntax (principled, block structured with no semicolons) and I really like laziness in general, because it makes it really easy to 'float out' and refactor code. OCaml has a much better module system, a very fast compiler, and is very well designed and thought out IMO. None of those are dealbreakers - it's just a matter of picking your poison.
"You might be surprised to find out languages from the 1970s and 1980s have better abstractions (e.g. functors/modules) than some designed today. :)"
...is uniquely appropriate as I've been amazed by and posted so much old work on forums that there's little that surprise me. Far as abstractions, I think Ten15 (below) was most interesting I found given its potential as an integrator. Burroughs Architecture, IBM System/38, Wirth's layered design of Lilith, Genera LISP's developer flow... the best attributes of these still haven't been matched imho by modern work. Still worth remembering and factoring into one's next project if possible.
That's a shame, they're excellent machines... I guess we'll have to find another sponsor soon.