Hacker Newsnew | past | comments | ask | show | jobs | submit | trgn's commentslogin

we need sqlite for llms

I think that we're getting there. I put together a workstation in early 2023 with a single 4090 GPU. I did it to run things like BERT and YOLO image classifiers. At that point the only "open weights" LLM was the original Llama from Meta, and even that was open-weights only because it was leaked. It was a very weak model by today's standards.

With the same hardware I now get genuine utility out of models like Qwen 3.5 for categorizing and extracting unstructured data sources. I don't use small local models for coding since frontier models are so much stronger, but if I had to go back to small models for coding too they would be more useful than anything commercially available as recently as 4 years ago.


im sure, but honestly, i would love to have a db engine that just writes/reads csv or json. does it exist?

DuckDB can do exactly this, once you get the API working in your system, it becomes something simple like

    SELECT \* from read_csv('example.csv');
Writing generally involves reading to an in-memory database, making whatever changes you want, then something like

    COPY new_table TO 'example.csv' (HEADER true, DELIMITER ',');

I wrote a CSV DB engine once! I can't remember why. For fun?

Microsoft actually provide an ODBC CSV data source out of the box.

Postgres can do that as well.

SQLite can do it

it's storage file is a csv? or do you mean import/export to csv?

> politician won't care

they dont care because nobody has a real gut feel affinity for computer programmers or the work, the sort of feeling that is required to animate somebody to action. it's never been a profession with any esteem, and the field never professionalized in the past 60 years, which is a shame, because we now see the outcome.


The web is a front-end environment, where users expect 60fps, but where developers violate pretty much all the rules.

> Zero-copy is the default, not the optimization:

the amount of fluffy mapping, destructuring, temp scope-creation, ... that is the norm now for JS/TS devs is excruciating. how did this become the norm? do it once it doesnt matter, but every single layer in the app doing this and things become jittery. you first take the hit at creation time, and then another time at GC. stop doing that! Pass references around to objects, not recreate them each time at some function boundary.

> Entity as pure identity.

Stop json.stringifiyng every thing! how many hashThing() implementations i have seen that are essentially Json.stringify. stop it!

> Cache locality by default.

a little less clear for web dev, much is missing in terms of primitives. but do think about it. if anything, it's good hygiene. fixed typed array does make sense, dont mix&match types in your array, ...

Save the web, think like a videogame dev!


Well, it's certainly not going to get better given that most of new Web code is by now probably written by LLMs. Which definitely aren't trained to write performance-oriented JS/TS.

lol another example of how things can always get worse.

The only "performance" that anyone actually cares about is the "P" in "KPIs".

Isn’t it funny, btw, how "performance" can also meant "acting out a fictional narrative"? Just a thought…


You forgot dynamic property creation/lookup instead of using a constant "shape" that JS engines can actually JIT optimize away.

And then there's the more detailed version of that where people write {x: 0, y: 1} in one spot and {y: 1, x: 0} in another and do not seem to realize that this under the hood they just sabotaged the ability of every JS engine out there to fully optimize any code related to it. Which also extends to situations where functions take objects that happen to share some properties as parameters: if you can put the shared properties first and in the same order in the object creation, it will result in better optimized functions.

(but tbh I think that as long as we don't fix the bloat that is tracking libraries first, all of this is optimizing the wrong thing)


> Pass references around to objects, not recreate them each time at some function boundary.

Non-primitives are always pass-by-reference. There's no mechanism to pass a non-primitive by value except edge-cases like giving ownership of a buffer to another process.

> destructuring

What about it? What backs the assumption that destructuring is inherently worse than dot and/or bracket syntax? Is there a behavior you think is unique to destructuring? Or maybe a specific report from one engine years ago?


> Non-primitives are always pass-by-reference.

it's a good thing. pass objects by reference.

> Is there a behavior you think is unique to destructuring?

depending on exact syntax, will collect values in another array or object. it's often used as the mirror-pattern of using named variables, which allocates an object for each function call.

in isolation these are not inherently wrong, at scale they start to add up. and should not be used in tight loops.


> pass objects by reference.

It’s not an option. That’s how JS must behave.

> depending on exact syntax, will collect values in another array or object

Not in JS. Maybe you are referring to rest syntax? That is not specific to destructuring i.e. functions accept rest parameters.

> which allocates an object for each function call.

No, in JS non-primitives MUST be pass by reference.


You're being pedantic and doing gotcha argumenting.

Pass objects by ref, I said that to underscore its better than create new objects. I get it, its the only way, but theyre still passed by ref.

Yes, [a,b,...rest] =... is restructuring and creates new object.

Named params create new objects, it's better to pass args individually, the ref creation under the hood is not comparibly impactful.


wrt (2) that is if satoshi had the foresight btc would ever blow up in the way it did. obviously, he had some intuition, remaining anonymous, but deliberately creating a fake trail does not seem super plausible to me

there was wage growth


Yeah, didn't people used to make like $10/week as the median wage at the turn of the 20th century? I agree that we have big problems now, but I feel like this analysis is deeply flawed without the inclusion of wage data.


Wage data, population growth, overall consumption, credit (and guarantees against it) are all drivers of inflation.

Look at student loans vs the cost of college:

1958: Federal program to encourage science and engineering. 1976: Remove restrictions on bankruptcy dismissal of this debt. 2005: Same rules for private loans.

Today college has a (as someone here so eloquently put it) a cruise ship ascetic, and has far more "administration" than "eduction" in terms of raw staff.

Tv went from an expensive box (fixed cost) to cable (monthly fee) to on demand programing (several monthly fees, and with ad's).

A phone used to be a single item in your house with a monthly fee. It was an item so durable that you could beat a robber with it and still call the police (see old att, black rotary phone). Now its an item per person in a household, that you can easily loose, might break if you drop it, and costs any where from 200 to 1500 dollars.

None of this is inflation in the traditional sense, but it does impact the velocity of all money in the system, and puts pressure on individual spending in a way that isnt even accounted for in this chart.

I wont even get started on housing, but I will leave this chart behind and ask those who care to point to the housing crisis on it: https://fred.stlouisfed.org/series/RHORUSQ156N


it's very expensive though. i used to live in philly close to 30th and had a reason to go up to nyc regularly close to penn, essentially perfect for taking amtrak, but ended up taking boltbus just because the price difference was very significant and time wise it was only like 30-45min slower.


Pro tip for those who like risk and are traveling regularly for non timely purposes they have dynamic pricing that rewards literal last 5m. I do Amtrak for like $15-40 NYC-PHL. You have to be signed in to the app otherwise they won't give you the sweetheart deal. Refresh reguarly the price changes constantly in the last 3-4h though I'll typically rock up to Penn and buy one 10m before.


Wish javascript devs would read this. If the web is slow, its because of them


Would sending voyager have been a real definite deadline?


Visiting this many planets was only possible due to a very rare alignment. It's a once a century event. That's why we sent two probes, not just one


Absolutely. You could wait decades or centuries for a useful planetary alignment.


Not really. Jupiter alone is good enough. Its huge mass accounts for almost all of the gain you get from any such slingshot. Launch windows from Jupiter to anywhere occur every 12 years. Voyager's alignment was captivating, but realistically if it hadn't happened, we would have just done separate Jupiter-Uranus and Jupiter-Neptune missions instead.


Odd to me that Elasticsearch isn't finding a second breath in these new ecosystems. It basically is that now, a RAG engine with model integration.


It’s definitely a use case for this and would’ve saved a lot of pain IMO but also seems like it would have added confusing technology to what was a VERY Python-heavy stack that would’ve benefitted from other elements.

Hardest part is always figuring out your company’s knowledge management has been dogsh!t for years so now you need to either throw most of it away or stick to the authoritative stuff somehow.

Elastic plus an agent with MCP may have worked as a prototype very quickly here, but hosting costs for 500GB worth of indexes sounds too expensive for this person’s use case if $185 is a lot.


ah got it! thanks for the color


The old joke Zawinski made about picking regex "and now you have two problems" applies here.

If you pick Elasticsearch, useful as it is, you now have more than two problems. You have Elastic the company; Elasticsearch the tool; and also the clay-footed colossus, Java, to contend with.


The people that survived it aren't willing to give it anymore of their breathing left


haha! it's been ok for me, but a lot of song and dance is required. the saas-version is a black box (in a bad way).


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: