> For example, they invented the .eh_frame walking technique to get stack traces from binaries without frame pointers enabled.
This is not an accurate summary of what they developed.
Using .eh_frame to unwind stacks without frame pointers is not novel - it is exactly what it is for and perf has had an implementation doing it since ~2010. The problem is the kernel support for this was repeatedly rejected so the kernel samples kilobytes of stack and then userspace does the unwind
What they developed is an implementation of unwinding from an eBPF program running in the kernel using data from eh_frame.
True, I should have been more specific about the context:
Their invention is about pushing down the .eh_frame walking to kernel space, so you don't need to ship large chunks of stack memory to userspace for post-processing. And eBPF code is the executor of that "pushed down" .eh_frame walking.
I believe this is a case of convergent invention – the idea of pushing DWARF/.eh_frame unwinding into eBPF seems to have occurred to several people around the same time. For example, there's a working implementation discussed as early as March 2021: https://github.com/iovisor/bcc/issues/1234#issuecomment-7875...
No, wrong decade and wrong split - the test & measurement equipment and scientific equipment was long gone from HP at the time of the HP -> HP inc + HPE split. It ended up in Agilent (1999) and from there Keysight.
HP semiconductors went HP -> Agilent -> Avago, now broadcom.
I've got two garages full of 80's and 90's HP lab equipment, and most of it even works. In that era, HP had the best hardware design/production capability in the world.
Unfortunately, in the same era, their software was almost always complete crap. I think the same rigid processes and controls that allowed them to make great hardware were the reason their software was awful. Their rigid processes made changing the software difficult, so it was harder for the devs to improve (and they usually didn't bother).
How do get the weights for the right set of experts for a given batch of tokens into fast memory at the right time?
The activated experts is only available after routing, at which point you need the weights immediately and will have very poor performance if they are across PCIe
Once your model is large enough you'll have to eat the offload cost for something, and it might as well be something where most of that VRAM footprint isn't even used. For current models, inactive experts arguably fit that description best. Of course, it may be the case that shifting that part of the graph to CPU compute is a better deal than paying the CPU-to-GPU cost for the active weights and computing on GPU; that's how llama.cpp does it.
Why is there a new kernel driver here at all? It appears that all it does it allocate system RAM (“DDR4”) and export it as a dmabuf for import to cuda as mapped external memory. Then a userspace shim hijacks APIs to use that if gpu memory is full. cuda already supports allocating mapped system memory, so AFAICT this could be implemented in the userspace shim with no new kernel driver.
Also as other commenters have mentioned, redirecting allocations to managed memory would also enable similar oversubscription
And the hijack approach only makes sense for making apps have this behavior with no changes, and could be done with minor app changes (e.g. PyTorch has a pluggable allocator interface). App changes also enable intentionally placing specific allocations.
My impression is that this is vibe from beginning to end, starting from a design that only makes sense if you are hallucinating
This is not about reclaiming memory by swapping the contents out to disk. It is about killing processes due to having overcommitted beyond the available memory plus swap space. The processes thrown out of the plane (targeted by the OOM killer) cannot be resurrected
Fair enough. I still think that the analogy is a bit overzealous given that my issue with the hypothetical weight-shedding strategy is ethical rather than technical.
It’s not a cargo cult if the actions directly cause cargo to arrive based on well understood mechanics.
Regardless of whether it would be better in some situations to align to 128 bytes, 64 bytes really is the cache line size on all common x86 cpus and it is a good idea to avoid threads modifying the same cacheline.
It indeed isn't, but I've seen my share of systems where nobody checked if cargo arrived. (The code was checked in without any benchmarks done, and after many years, it was found that the macros used were effectively no-ops :-) )
> even on x86 on recent server CPUs, cache-coherency protocols may be operating at a different granularity than the cache line size. A typical case with new Intel server CPUs is operating at the granularity of 2 consecutive cache lines
I don’t think it is accurate that Intel CPUs use 2 cache lines / 128 bytes as the coherency protocol granule.
Yes, there can be additional destructive interference effects at that granularity, but that’s due to prefetching (of two cachelines with coherency managed independently) rather than having coherency operating on one 128 byte granule
AFAIK 64 bytes is still the correct granule for avoiding false sharing, with two cores modifying two consecutive cachelines having way less destructive interference than two cores modifying one cacheline.
Original iPods (and early iPhones) weren’t locked down as much. There were a number of utils that could manage your library. ml_ipod plugin for WinAmp comes to mind.
This was in the “truth” posted by Trump on his social media announcing the deal:
> It is my Great Honor to report that the United States of America now fully owns and controls 10% of INTEL, a Great American Company that has an even more incredible future. I negotiated this Deal with Lip-Bu Tan, the Highly Respected Chief Executive Officer of the Company. The United States paid nothing for these Shares, and the Shares are now valued at approximately $11 Billion Dollars. This is a great Deal for America and, also, a great Deal for INTEL. Building leading edge Semiconductors and Chips, which is what INTEL does, is fundamental to the future of our Nation. MAKE AMERICA GREAT AGAIN! Thank you for your attention to this matter.
The president has been known to not know all the facts or exaggerate about what is known. Personally, and sadly, his tweets are worthless than my fortune cookies.
There is an important difference between these scenarios:
1) A member of the opposition party tweets "The president stabbed a kid" without any proof. I go on facebook and post "WTF why did the president stab a kid? He is so evil."
2) The president tweets "I just stabbed a kid" without any proof. I go on facebook and post "WTF why did the president stab a kid? He is so evil."
In general its a good rule to avoid using any politician's quote as fact. Especially at the federal level, they've all made a career of exaggerating and telling partial truths to earn media coverage and votes.
Let’s not “both sides” his behavior. This president lies about everything, and actively causes harm by lying maliciously about people he would like his followers to target.
I'm not "both sides"-ing it. Presidents all lie frequently, I'd argue they lie about most things. Without knowing what the truth actually is we would have no way of knowing who lies more, and at the end of the day my concern is with them lying at all rather than to what degree they lie to the public.
There is quite a difference between. exaggeration and blunt lies.
Also most exaggeration happens during campaigns for getting votes, but rarely the result is a strong enough mandate to push all things through, thus one has to compromise ... but campaigning on "well, realistically my options will be limited" doesn't really work, especially as the campaign promises form the negotiation base lateron.
What you point to is an odd reversal for sure. Trump is actually doing many of the things he campaigned on while most candidates lie during the campaign. Trump now lies about seemingly obvious or unimportant things now in office, where many presidents either wouldn't waste a lie on something unimportant or wouldn't bother acknowledging the topic at all.
They all still lie though. Whether a particular lie can be considered an exaggeration boils down to how strict a line one draws around what a lie is. To me, if a president speaks only a partial truth or a misrepresentation if information they very much have access to, its a lie.
He told the truth, but also many lies in parallel. Where is the wall that mexico paid? Where is the peace in Israel and Ukraine? 6 days are over. The truths in between this are useless.
Can you find a president in recent history that didn't fall short of many or most of their campaign promises?
I don't say that to defend trump, the guys is a narcissistic asshole. I only say that to point to the fact that he's doing what any other politician does - say what it takes to get elected then play the game that makes you the most money.
Are you sure congress didn’t authorize this ? i.e. actually specified that the money could only be used for grants and could not be used for equity purchases?
> The Department of Commerce is authorized to provide funding in various forms, including grants, cooperative agreements, loans, and loan guarantees, in exercising its Section 9902 authorities
AFAICT the relevant section of law says it is up to the Secretary of Commerce to determine the funding type to be used for the semiconductor financial assistance
This is not an accurate summary of what they developed.
Using .eh_frame to unwind stacks without frame pointers is not novel - it is exactly what it is for and perf has had an implementation doing it since ~2010. The problem is the kernel support for this was repeatedly rejected so the kernel samples kilobytes of stack and then userspace does the unwind
What they developed is an implementation of unwinding from an eBPF program running in the kernel using data from eh_frame.
reply