Looks like open-source is just a marketing tool for AI companies before they have a good enough model to sell. I guess we have to look for what Meta is going to do with LlaMA 3.
How is this a problem? So many companies have been founded around premium versions of open-source products. It's good that they've even given us as much as they have. They have to make the economics work somehow.
It’s a significant problem when “Open Source” is used as an enticement to convince people to work on and improve their product for free, especially when that product inevitably relicenses that work using a sham of a “rewriting” process to claim ownership as though it voids all the volunteer’s efforts that went into design, debug, and other changes, just so that source can be switched to a proprietary license to make the product more VC/IPO friendly. And all of that cuts the knees out of the companies you claim it created in order to capture a portion of their profits despite the fact that they most likely contributed to the popularity and potentially even the development, and therefore success, of said “Open Source”.
IMO, it is just a new version of wage/code theft with a “public good” side-story to convince the gullible that it is somehow “better” and “fair”, when everyone involved were making money, just not as much money as they could be taking with a little bit of court-supported code theft and a hand-waive of “volunteerism”.
The people who use these open models are doing it because they find them useful. That's already plenty of benefit for them. The "ecosystem play" of benefiting from volunteers' mods to open models is certainly a benefit for the model trainer. This fact doesn't eliminate the benefit of people being able to use good models.
It's not a problem from a moral perspective or anything - we all know these models are very expensive to create.
However, from a marketing perspective - think of who the users of an open model are. They're people who, for one reason or another, don't want to use OpenAI's APIs.
When selling a hosted API to a group predominantly comprised of people who reject hosted APIs - you've got to expect some push back.
Is this true? I know a whole lot of people that use and fine tune Mistral / variants and they all use OpenAI too. (For other projects or for ChatGPT)
From my perspective, I want to use the best model. But maybe as models improve and for certain use cases that will start to change. If I work on a project that has certain parts that are fulfilled by Mistral and can reduce cost, that's cool.
I'm surprised how expensive this model is compared to GPT-4. Only ~20% cheaper
> I'm surprised how expensive this model is compared to GPT-4. Only ~20% cheaper
I'm guessing all currently available paid options are operating at a (perhaps significant) loss in order to capture market share. So it might be that nobody can afford to push the prices even lower without significant risk of running out of money before any "market capture" can realistically be expected to happen...
This. Also, at least be upfront with users about motives. OpenAI stopped claiming to be "open" about 2-3 years ago. That's fine—at least I know they're not pro-OSS.
But Mistral has been marketing itself as the underdog competitor whose offerings are on par with gpt-3.5-turbo and even gpt-4, while being pro-OSS.
I've been saying this for months but every time I get down voted for saying it. It annoys me that people fall for these marketing tactics and keep promoting and advertising the product for free. It's not just the models though- even tools that started off as open source ended up aiming for VC and stopped being totally open.
Examples: LlamaIndex, Langchain, and most likely Ollama.
Whoever is lagging will be open source. It's why AMD open sources FSR but Nvidia doesn't do the same for DLSS. There is nothing benevolent about AMD and nothing evil about Nvidia. They are both performing actions that profit maximize given their situation.
> They are both performing actions that profit maximize given their situation.
That really rings like moral relativism. Even 15 years ago when we were still talking about "GPGPU" and OpenCL seemed like a serious competitor to Cuda, NVidia was much less open than AMD. Sure you can argue that they are "just" profit maximising, turns out it's quite detrimental to all of us...
If what you're saying is that we shouldn't be naive when dealing with for-profit companies and expect good gestures, I agree. But some are more evil than others.
It isn’t moral relativism. It’s just economic sense. In both cases.
There is no moral requirement to be open source.
Being closed is not fraud, coercion, theft, dishonest, anti-competitive, …
(On the other hand, being open, in situations where closed would be more profitable, is taking the moral high ground.
Open provides better value for the customer, user, and community.)
Aside from moralizing, the economic puzzle is: How to align the economic incentives of businesses with the real long term community value of openness. While also providing greater resources to successful innovators to incentivize and compound there best efforts.
(Note that copyright has been the solution to this problem for cultural artifacts. And patents try to do this for tech, but with more problems and much less success.)
The community needs to train its own models, but I don't see any of that happening. Having the source text would be a huge advantage for research and education, but it feels totally out of reach.
It's funny how people are happy to donate to OpenAI, that immediately close up at the first sniff of cash, but there doesn't seem to be any donations toward open and public development, which is the only way to guarantee availability of the results, sadly.
I should add: Mistral, Meta, etc don't release open source models, all we get is the 'binary'.
Those initial OpenAI donations really were for open development.
The problem was, there was no formal legal restrictions put in place at the start that stopped them from hatching a private subsidiary or not remaining open. Just that the initial organization was non-profit and for AI safety.
Which is the only way that could have been stopped.
A failure of initial oversight. A lack of “alignment” one might say.
The cash required to develop and train the models makes the open-source approach challenging, if not impossible, for companies who don't have another business to support it. You need to be Meta - with a huge cash cow - to have the option to give away your work for free. After all OpenAI tried and came to the conclusion that it couldn't succeed as a pure open-source non-profit company no?
Especially as the model weights are literally a huge opaque binary blob. Much more opaque than even assembly code. There is plenty of precedent for what "open source" means, and these aren't it.
Edit: not that I mind all that much what they're actually doing, it's just the misuse of the word that bristles.
Open source means "the preferred version for modification" and this fits with model weights since you can fine tune them with your own data. Modifying raw training data would be quite unwieldly and pointless.
Isn't this comparison completely backwards? As I understand it, it's useless for a person to own a source dataset for an LLM, because its "compilation" costs $n million.
Why would a crowdfunded ai project need to be in Japan particularly ?
But regardless, part of the answer might be that it might be more attractive for "capable people" to get serious money working for a for-profit AI company at the moment.
