Very interesting... my first (and last) encounter with heterogeneous OSes was back in the mid nineties with TaOS (best reference found online is http://www.uruk.org/emu/Taos.html but the website is rather wonky).
Honest question: I’m not the only one who confused this for the Popcorn Time torrent streaming player, am I?
My first thought was: wow, they took this idea so far now they made an OS out of it.. I imagined an OS preconfigured with torrent clients, easy-to-use VPN client, fancy Netflix-like media center...
It's a single system image [1] platform built with Linux. The main idea being that the OS is running on multiple computers while presenting a single computer abstraction to the other software running on it. You write your program as though it's running on a monolithic machine, and the SSI platform manages it across multiple distributed machines.
The concept is more popular in HPC & scientific computing than it is in mainstream IT.
This is off topic, but it's proving hard to google.
What's the significance of the hand gesture the women in the video makes at 1:13 and again at 1:39?
It is a prevalent hand gesture in the Indian subcontinent. Could mean a lot of things, and interpreted along with the facial expression. In this case, it means "That's pretty cool", looking at her facial expressions.
It could mean "What the heck is going on?", when accompanied with a different facial expression.
I judge the video not by the enthusiasm of the people involved, but by the actual contents of the video.
You know what is an important workplace skill (tech or not)? Speaking intelligibly.
Some people seem to think that all criticism of accents is bad, because racism. This is actually backwards. I myself am an ESL speaker, and therefore have some trouble understanding heavy accents. Using people with heavy accents in a video instead excludes people who speak English as a second language, since we, as a group, are not trained in it. Our experience of spoken English have mainly come from TV and movies, where people speak clearly. So when we see something like this video, we are effectively excluded from understanding it.
Saying things like “was it really that hard” is condescending. Yes, it’s hard if you don’t have a lifetime of training, which people in English-speaking countries do. We ESL speakers don’t. Producers of videos should be more considerate.
I'm not sure that's entirely accurate. I know I have an easier time understanding History on Fire[1]'s thick Italian accent than Americans do. When learning English, you learn the correct pronunciation and also the "incorrect" pronunciation used by your classmates.
None of the speakers in the video use my local version of what passes for English, and therefore I have no training in understanding the video. It is difficult for anyone who has not lived with people having all of those accents. Which is unlikely if you’re living in a non-English speaking culture.
They speak perfectly intelligible English, you just need to listen for a second to understand the accent. Subtitles would be great but come on, there's no reason to be a jerk about it.
Beowulf had no shared memory abstraction, programs had to be written using message passing, e.g. MPI.
I'm not convinced a shared memory abstraction is a good thing for distribution across machines. Some Beowulf clusters actually had much lower latency interconnects than ethernet, e.g. Quadrics, but we still needed to carefully optimise the traffic. With a shared memory abstraction, such optimisation is out of our hands.
Ah! I've been saying we should do this for years now. Everyone in the cloud space is reinventing the wheel when what we need is a cloud OS that abstracts away multiple computers. I can see a bright future for a project like this.
I can't find it but that sounds like the model a platform company that works with CCP to host EVE online makes. They program one game but they shard different sectors of the universe out to hardware based on demand.
They either host it or it was just a tech demo they did with eve online. Wish I could remember it was a super cool job advert, not my skill set though.
In mainstream IT, you'd usually set up a Hadoop or Spark cluster on the machines and then write your code to run against one of those. What's the advantage of this approach over those?
Automatic vertical scaling without changing applications.
One needs to rewrite apps in order to get the advantages of Hadoop/Spark/MapReduce in general, and not all problems are solved by simply doing that.. It is a different thing/paradigm, really
> The Popcorn Linux project is exploring how to improve the programmability of emerging heterogeneous hardware, in particular, those with Instruction Set Architecture (ISA)-diverse cores, from node-scale (e.g., Xeon/Xeon-Phi, ARM/x86, CPU/GPU/FPGAs) to rack-scale (e.g., Scale-out processors, Firebox, The Machine), in both native and virtualized settings. Additionally, the project is exploring how to automatically compile/synthesize/execute code on ISA-heterogeneous hardware.
Let's break it down! (I'm no expert so sorry for any mistakes, and this is simplifying a lot)
> improve the programmability of emerging heterogeneous hardware
reducing the fiddlyness of developing for mixed hardware
> those with Instruction Set Architecture (ISA)-diverse cores
those with multiple CPUs speaking different machine code
> in both native and virtualized settings
both bare-metal and VMs
> automatically compile/synthesize/execute code on ISA-heterogeneous hardware
automatically translate between the different machine codes.
---
Now for what it actually means, is that Popcorn is a single Linux that can simultaneously run on a bunch of different processors (say, your mobile phone, a raspberry, and a desktop) all at once while being used just like a single Linux system to the end-user/programmer. I don't think it's meant to be used by consumers, but for servers or supercomputers, that kind of stuff.
> I don't think it's meant to be used by consumers, but for servers or supercomputers, that kind of stuff.
Why would servers/supercomputers care about heterogeneous hardware (including multiple ISAs, even!)? If I'm playing around at home, that's great; I'd love to make a cluster out of a couple of old laptops, an old phone, and a Raspberry Pi. But at work, I'd just... buy a dozen identical servers. Even across generations, x86 is mostly uniform enough to just ignore the changes (with some minor gotchas around extensions, but if you only migrate forward that should be fine too).
A supercomputer cluster wouldn’t care. A cluster of supercomputer clusters might.
Know those moments in movies where they put “all the computers in the world” onto the task of cracking some encryption before it’s too late? This could be used for the technical part of that (once you’ve got all the HPC labs’ cooperation.)
Think of it in a "Compute at home", "folding at home", blockchain-sense then latency doesn't matter as much, I do not know if anyone has tried solving things with big clusters separated by 40ms-60ms latencies.
Maybe if you had some kind of amazing cache system that organized everything into one big tree so that data needed for different computations is as local as possible. It would be very challenging though.
at Cray, at one point we had custom vector processors, custom SMT(for lack of a better term) processors, FPGAs on the roadmap, GPUs, and commodity AMD processors...and a strong desire to ship them in the same frame
that may have been the only time in history though
I imagine it can't (that would be way too good, everybody would be using it already). I also don't think there are many examples of supercomputers with diverse ISAs today.
But a system like this is a requisite for non-uniform computers to exist. Thus it must exist without the other, at least for a while.
It looks like a multi-node version of Linux - a bit like a MOSIX or something - but without the limitation that the hardware needs to be the same on each node. E.g. you could have one X86, one ARM, one FPGA, and a single app could deploy threads across each of those "CPUs".
Equally, I guess it looks a bit like a Java application server in terms of function - applications appear to be compiled to whichever hardware target the application needs to run on.
Utility infrastructure like shared memory etc appears to be part of the design, so the underlying hardware systems are as transparent as possible.
"The project is exploring a replicated-kernel OS model for the Linux operating system. In this model, multiple Linux kernel instances running on multiple nodes collaborate each other to provide applications with a single-image operating system over the nodes. The kernels transparently provide a consistent memory view across the machine boundary, so threads in a process can be spread across the nodes without an explicit declaration of memory regions to share nor accessing through a custom memory APIs. The nodes are connected through a modern low-latency interconnect, and each of them might be based on different ISA and/or hardware configuration."
Intuitively, I would think that the way to do this would be as follows:
First, we have the memory hierarchy, disk at the bottom, increasingly faster (but smaller) regions of memory closer to the CPU.
Now, if let's say a process shares memory across nodes, then what seems to make sense would be moving chunks of that memory, as needed, as "cache blocks" to the local machine that needs it.
Observation: That pattern is really not very different than what OS's do with swap files / virtual memory, except that the "swap file" is actually a larger memory region located on a remote node, across the network...
Of course there may be other issues with respect to locking, synchronization objects in memory, thread scheduling, etc., etc., but if you got the above memory access pattern right, then you'd go a long way to enabling such an OS...
Can you please stop putting "Excerpt" in your comments like that? It's enough to use quotation marks or ">" or some other visual marker to show what you're quoting.
The trouble with the way you're doing it is that it signals that the entire comment is an excerpt, and since you've done it a bunch of times, that makes your account look like a single-purpose account or a bot, neither of which are allowed here. I was going to post a totally different reply asking you not to do those things, and then I noticed that actually you're posting normal replies with a nonstandard header.
I would like to profusely apologize for using "Excerpt" in my comments for places I excerpt other content, although, I am hoping you'd understand that the reason I do that is for my own way of organizing my own notes... Yes, I understand that HN is a public forum, so in order to please the forum rules and regulations, so from this point forward I will try hard not use "excerpt". Even though I will try not to, if I do it in the future, then understand it was an old habit ingrained upon my stiumulus-response cortex, but nonetheless, I will make a concerted effort not to.
You're an awesome HN user. Don't sweat it for one second. Thanks!
If you simply put "> " in front of what you're quoting, it signifies the same thing. Or you can put it in double quotes, or put asterisks around it and it will get italicized.
I note some comments saying that this is more for HPC. I’ll point out that most mobile ARMs have different kinds of cores on the same die (the so-called big.LITTLE approach), and of course often have utterly different functional units (GPU) as well.
Joomla has never been great compared to its competitors. It's been many years since I've even heard the name. The are tons of better cms platforms no a days
The project seems still alive and active, at least according to its recent RFC'ing from LKML [0].
For some more information, Phoronix also recently wrote an article [1] about it, also the GitHub repo [2] seems to be somewhat active (last commit was 06-02-2020).
Ah, you're right, it does appear to be still alive. If anyone from the popcorn linux team sees this, please consider updating the outdated portions of your website (and getting an HTTPS certificate). Had I never read this comment I would have gone on unaware that the project was not dead.
Honest question: I’m not the only one who confused this for the Popcorn Time torrent streaming player, am I?