As someone who works on the foundations of quantum mechanics (how to formulate, why we think it's right, etc.), I'll disagree with this. The fact that much of the detail from many experiments is available somewhere is not much more useful than being told that code is available from the authors if only you email them and wait 3 months. The barrier to figuring out this stuff are immense.

Furthermore, it's not necessary to communicate the actual, usually circuitous, route taken historically by scientists. Instead, you could just describe the series of hypothetical simplified experiments whose results would lead one to quantum mechanics and rejecting alternatives. This is never attempted in a serious way (trivial uses of the Stern-Gerlach experiment as a model of quantum mechanics not withstanding).

Dirac derives quantum physics from first principles, including the relevant thought experiments, in one of his books. that was good enough for me. made a lot of sense.

"Principles of Quantum Mechanics"

Dirac's book is a perfect example of what I'm talking about. Just think about the fact that this book from 1958, which makes a valiant attempt at justifying a few parts of the formalism (it's on my bookshelf), is basically a high-water mark even though it's very flawed, especially given what we know today. What does it tell us about the ability of physicist to transmit ideas to the next generation with high-fidelity if, to understand why the quantum formalism is how it is, you recommend reading a 60 year old book by a guy who was alive when it was being formulated!

Some of those flaws:

(1) Dirac postulates, as most people do, that measurable observables are to be identified with Hermitian operators. This is a mistake that can be traced back to von Neumann. In reality, the larger class of normal operators are perfectly fine as measurable observables. Indeed, measurements are properly associated with only an orthonormal basis, and it is completely unnecessary to label them with eigenvalues, real or otherwise. (To see this, just observe that there is no physical difference between experiments that measure x and x^3 for the position of a particle.) Dirac's discussion on page 35 is just wrong.

(2) Unless my memory fails me, Dirac gives little to no justification for why we use tensor products to build up the state space of a many-body system from the state space of a single-body system, a tectonic shift from classical mechanics at the heart of the weirdness and power of quantum mechanics.

(3) Dirac was written before Bell's inequality. I mean, just look at the pithy discussion by Dirac (p. 4-7) to justify fundamental indeterminacy, one of the most profound things we know about the universe. Do you think this would have convinced Newton? Or Dirac in 1925? (We know it didn't convince Einstein.) This sort of thought experiment is lovely for an article in Scientific American to give laymen a sense of where things come from, but it's nowhere near the rigor with which we should teach physicists.

I really wish I could find a book (or series) that started with explaining the (non-)results of the Michelson-Morley experiment, and traced through the major experiments that underpin modern theory. Just picking out the major quintessential experiment (types) -- things like the Bell test.

Much of science is really just subtle modifications or tests of major theories, but I feel like you could write up... say 12 keystone experiments in 100 page summaries and publish it in 2-3 volumes, (eg, QM and relativity volumes).

The problem I have with only presenting "polished" results is that we lose the context of our modeling -- eg, QM seems to have included non-determinism in an effort to preserve locality, but locality couldn't be preserved in the face of other results, so is non-determinism merely an extra assumption included for legacy reasons (eg, technical debt because no one wants to clean up the model/people forgot why we included it)?

You'd never even think to ask that question if you only saw the cleaned up model of QM divorced from its (philosophical) roots.

The vast majority of research papers are barely readable by people in the field, say nothing about a beginner.

research papers can all be read by "people in the field".

its just that someone who is doing environmental physics doesn't necessarily understand papers on string theory. science is specialized, yea. if you want to understand research papers, get an education.

or just do something else. cutting edge research is not for everyone. not everyone is supposed to understand it. whats next? people demanding research papers be dumbed down to adhere to the freedom of information act?

science is hard. big deal. you cant have all the things at the same time. read the hawking books written for casuals if you want a cute little story about science.

> research papers can all be read by "people in the field".

Not necessarily. The classic example is Shinichi Mochizuki's work.

He's done some incredible work but he's basically invented his entire field out of thin air, he doesn't publish frequently, and the papers he does publish are essentially impenetrable.

He likes it this way and doesn't want to "dumb down" his work for the mere mortals who need to review it. That's essentially the end state of your argument - if you can't comprehend it you're not "in his field" and you're obviously not qualified to discuss it. And he's certainly not going to waste his time teaching some dunces the basics of his field.

https://www.newscientist.com/article/dn26753-mathematicians-...

https://www.newscientist.com/article/2099534-mathematicians-...

My point is the poor quality of the writing, not the obtuseness of the science. Theoretical physicists and mathematicians aren't novelists.

Writing in a clear way, without drammatic grammatical errors, is not "dumbing things down".

For the casual reader, the "bad writing" aspect of the difficulty is severely outweighed by the "lack of domain knowledge" aspect, by several orders of magnitude. If you take a random modern research paper, and put it through a handful of editors to make it superbly written in a clear, convincing language that gets its points across very easily, the casual reader uneducated in the background knowledge will barely be able to tell the difference.

A large part of any scientists job is communicating results. The people you criticize may not be novelists, but they are certainly professionals when it comes to communicating technical knowledge.

However, this knowledge is only communicated to other experts in the same - often very narrow - subfield. Often the definitions that give you a hard time when reading a paper have been refined over several years and are basically known to all other people working in the same field.

This is not ideal, but there is simply not enough manpower to produce good and generally accessible summaries of current research topics every few months...

> The people you criticize may not be novelists, but they are certainly professionals when it comes to communicating technical knowledge.

No, that's the point I'm trying to make. They do not have enough training in making themselves understood. Hell, a good portion of them don't even have the language in which they are writing as their primary language.

Plus, it seems that authors sometimes feel compelled to appear smarter by leaving out the intuition and motivation behind a result, and in particular the easier and more intuitive simple cases (which might have been the inspiration for the result in the first place), and instead present only the most abstract & most general version of the result that they managed to prove.

Including some of the enlightening historical path towards that result is not "dumbing things down".

show me one paper that you cant understand because of the grammar but would be perfectly fine reading it if only the lyrical style was a bit more up your alley.

Effective communication is about far more than whether the reader/viewer can eventually understand the material. Even basic presentation skills can make a huge difference both to the speed at which someone can receive and understand new information and to how well they will retain that information later.

Unfortunately, many academics receive little if any training in good presentation before being expected to lecture or write at undergraduate or graduate level, and while some are naturally gifted presenters anyway, most inevitably are not. Consequently, many career academics have no idea how poor their presentation skills are, how ineffective their presentation is as a direct result, or how much better they could be. They just get stuck at a very low level, but without the kind of introductory/remedial training that would be given to someone whose career involved presentation skills in the professional world. And of course if anyone with broader experience dares to suggest that there might be room for improvement, the instinctive reaction is denial.

A little irony is that some of the most engaging and informative presenters I have ever seen or read come from that same community, but despite the emphasis on peer review in their research work, when it comes to soft skills the weaker presenters typically have no idea how bad they are and therefore make no attempt to learn from their stronger colleagues and improve.

i've only had exceptional teachers in university

Unfortunately, as your own choice of word "exceptional" implies, most are not so lucky.

I once sat in a review meeting at the end of a year with members of the faculty responsible for teaching collecting feedback from many of the undergraduate students. When challenged about the poor quality of many lectures, the response was essentially that they can't make the lecturers go and learn how to lecture competently because the lecturers wouldn't stand for it.

Try that attitude in the professional world and you'd be in a remedial programme on your way to getting fired.

Fortunately, I expect that with the advances in modern technology and changes in modern careers, the old-school universities that think a famous name and charging high fees mean they can get away with anything will soon be obsolete, and so will the incompetent parts of the academic community sheltering within them. They will need to find new ways to offer dramatically more value than interested people can find on their own with all the modern resources we have available, or they won't be able to justify people taking several years out of their lives and paying a fortune in fees to attend any more.