We haven’t tried SQL specific models actually. That’s quite a good idea. Will try it out.
Yeah it mostly seems to be inconsistent at generating the SQL queries themselves. I guess my question was if this is even a good way to do it in the first place, or if we’re better off trying to analyse the data in a different way
I understand the push to use the app for collecting my data for ads and what not but good lord is it too difficult to keep the web channels open? Why is this not available on browser :(
I've seen the power of moving systems to more machine readable file formats such as Parquet, Arrow as opposed to storing them as csv, json etc. For people who are actually making these design choices in my experience it has always been a better idea to prefer smarter formats over readable ones for large scale systems. They really help in longer term for :
- cost (more maintainable from the tech overhead required).
I've been seeing a sharp dichotomy between the data source originators and the ones processing it. For the folks that originate data, Excel seems to be such a native language that it is hard to move too far away from it. For processing, absolutely consider parquet or similar.
Such that it really depends who you are talking to about which format you should use. Ideally, it seems, support both. Allow rapid ingress and egress of data to/from Excel in whatever way that you can. CSV is the common stopgap between the two, with loads of sharp edges on loading it into Excel.
Having worked with some teams/stakeholders who have programs and SOPs that are very tightly coupled with the excel, csv data handling I do have a few observations:
- These processes and SOPs are actually quite optimal for low scale use-cases but as organizations grow they find themselves in a tough space where even after you have a hiring spree going on everything seems to be falling apart and not maintainable.
- You almost always do lose out on an audit trail. Now, you don't always need an audit trail but if you find your org sharing tons of excel docs being shared on a regular basis you do need a tech integration to solve it. The lack of it means a gap in identifying things that can be made simple and no way to identify / fix things that may have gone wrong over a period of time.
- I fully agree with the statement
. Ideally, it seems, support both. Allow rapid ingress and egress of data to/from Excel . The data in systems in software systems is useless it's almost always meant to be consumed by humans (mostly). And that just means while I do most of the authoritative processing in file format that computers do well with I allow it to be converted into a more human readable format which happens to be excel in some cases and visualizations in others.
Fully agreed on the problems of scaling up places that share data using Excel. I harbor a fear that there is no good way to share data in a large organization, and that you are best looking for ways to shrink the organization and reduce the number of people that need to be involved.
Same thing, largely, seems to apply to documents. Yes, you could farm out a book to a hundred authors and editors. We could build a tool that allows active collaboration that shows what everyone is doing. And we even have some exciting data structures that seem to make this a viable path forward. My experience has been that that isn't the case, though. Specifically, the data structures are fine and nice, but getting even a dozen folks working on the same doc at the same time usually reduces the document to a glorified database with each person doing an optimistic lock on their part so that they can do what they want to do.
While it's not the best experience I've been using iPad for coding for over 2 years on my personal projects. My setup uses a machine at home with
- https://github.com/coder/code-server
- terminus - for playing with terminal work
- tailscale - for VPN and accessing local servers on home server
I must admit not having a ESC key in ipad keyboard is a pain as I'm a heavy vim user but after a few weeks you kind of get used to CMD + "." for escape and life moves on. IMO, the biggest pro I have for using iPad is the fact that the size and battery life as the heavy computing happens on my home server plus the small network footprint on mobile network is just helpful.
Have you tried fiddling around with temperature or SQL specific finetuned models?