Neutrino speed must be subjected to the "scientific method"

"Not so fast, neutrinos"

MIT physics professors examine the subatomic speed limit controversy

by

Stephanie Holden

October 14th, 2011

The Tech

On Sept. 23, European scientists announced that they had observed neutrinos, a class of subatomic particles, traveling faster than the speed of light — the universe’s fundamental “speed limit.” The experiment, OPERA (Oscillation Project with Emulsion-tRacking Apparatus), was a collaboration between the Italian Gran Sasso National Laboratory (LNGS) and Europe’s high-energy physics laboratory CERN. Since the announcement of this anomaly, the scientific community has been hotly debating its validity, as well as the possibilities that could arise from such results.

MIT Physics Professor Scott A. Hughes said, “Carl Sagan had this saying, that extraordinary claims require extraordinary evidence. This is not extraordinary evidence.”

Hughes pointed out that the OPERA experiment was not originally designed for measuring the speed of neutrinos. The main goal was to transmute — or convert — one type of neutrino, called a muon neutrino, into a tau neutrino, a heavier type of particle.

Hughes conceded that the researchers “have done as good a job as they can, but this is extremely hard to measure … there’s this table of systematic errors in their measurements, and one error tends to dominate. If they combined their errors in a different way, their results could have been within the error bars.”

“I don’t think the paper is outlandish,” added Physics Professor Janet Conrad.

However, she and Physics Professor Frank Wilczek — a Nobel laureate — both said that the main evidence that contradicts OPERA’s result is the data set from Supernova 1987A, when neutrinos produced from the star explosion arrived only a few hours before the light did (neutrinos leave the dying star before visible light from the explosion).

Every supernova is accompanied by production and emission of a massive quantity of neutrinos. Physicists can calculate the relative time between when the neutrinos are emitted and when the light is emitted from the explosion. If OPERA’s results are correct, however, the neutrinos should have travelled faster and arrived a few years before the light.

Hughes and Wilczek both guess that scientists will most likely approach the neutrino announcement with a variety of new experiments and do tests with different baselines.

“I would say there’s a 98 percent chance this is a systematic error,” Hughes said.

Scientists note that other predictions of special relativity are valid, and that these neutrinos might be something “special and weird.”

“Within the theoretical framework, we have been very successful in other parts, which is why it’s hard to isolate this neutrino ‘disease’ in this small sector,” Wilczek said.

But no one is claiming that the European researchers were careless. On the contrary, Wilczek believes that all of the scientists are “competent, professional experimenters who have been wrestling with this for months and can’t make this go away.”

“I’m not claiming they’ve done it wrong, I’m saying that it needs looking at very carefully,” added Hughes. “There’s a big difference between precision and accuracy — you can measure with precision a very inaccurate result.”

What if it’s true?

If it is true that neutrinos can travel faster than the speed of light, fascinating new lines of inquiry could open. One theory is that these speedy neutrinos could be a crack in the universe that reveals extra dimensions in high energies.

“If this were correct, our GPS wouldn’t work,” Hughes said of the navigation technology that relies on relativistic principles.

“It’s premature, to say the least, to speculate wildly about the implications, either theoretically or technologically,” said Wilczek.

Talk in the media about traveling in time or having causal loops were a misunderstanding of relativity. “If there’s any limiting speed, even if it’s not the speed of light, one would not be able to close the loop from the future to the past,” Wilczek added.

Although he is doubtful of the results, Hughes does not have any criticism of OPERA’s report itself. “The paper is very clear — they say they’re throwing [this discovery] out there for further testing. Their paper is very careful and pretty conservative.”

He found it “irresponsible,” however, that the authors held a press conference immediately after their accidental discovery. In fact, some of the researchers who were part of OPERA actually removed their names from the paper because they found the analysis to be too preliminary to be able to release the results in such a manner.

“This is one of the few things that reveals the tension that was going on within the experiment,” Hughes pointed out.

Public reaction

The fact that this story has made a huge appearance in headlines over the past few weeks does not surprise Hughes, but he is worried that after it dies down, any future and possibly contradictory discoveries will not have as large an impact in the media.

“No headline will say, ‘Oops, we goofed.’ … My concern is that this potentially big splash will not be compensated for by correction,” said Hughes.

Physics student Asher C. Kaboth G also noted that “It’s harder for the general public to understand the little details … it’s difficult to explain.” Those who do not know much about special relativity might have misconceptions about what these results mean and their possible implications, he said.

Conrad did not like the way some physicists reacted to the news. “Way up there in the responses I don’t like [is], ‘If it doesn’t fit my theory, it must be wrong.’ That’s not okay to tell people.”

She said that if neutrinos really do travel faster than the speed of light, it breaks current theories and scientists will have to construct new ones.

“There’s a difference between us saying what nature will do and nature telling us what it does,” Conrad said. “When you find a violation, you have to find a way to put it in the perspective of other data.”

Despite concerns, the neutrino results have been a great teaching opportunity for many professors. In Hughes’s 8.033 (Relativity) class, he discussed the concept of the experiment in lecture and asked students to think carefully about whether they believed that neutrinos could travel faster than the speed of light.

“In the hands of someone who can discuss this well, and the ears of students open to listening, it’s a great topic,” Hughes said.

Conrad also brought up the subject in her 8.02 (Electricity and Magnetism) class and asked her students to answer questions like “If this result is proven wrong, what does this say about science? Does science ever get it right?” Her opinion is that things go wrong in science all the time, but the beauty is that one discovery leads to the next, and we continue to change what we know about the world.

Wilczek agreed that on the whole, it’s a good thing that people are noticing current research in physics, and that there is exposure of the scientific process. “There’s something about Einstein and space that even after all these years has a certain magic because it’s so profound and unexplained,” he said.

"Theoretical physics...good epistemology?" poll

How much epistemological relevance is there in theoretical physics being that it favors philosophy more than empirical science?

A lot...2
Some...1
None...2

I am not the only one who has raised this question...so has Steven Weinberg who clearly states that theoretical physics is more philosophy than science with specific reference to "string theory" and an abandonment of the basic tenants of the scientific method and a bona fide source of knowledge.


Limit on understanding the universe

Michael Shermer--comments on "reductionism"

"Parallel Worlds, Parallel Lives"--YAWN

"Put a Little Science in Your Life"...please

Scientific Method out the window?

Sound science/philosophy...a product of sound thinking

Theoretical physics...philosophy/sci-fi?

Time to re-evaluate physics

Scientific Method out the window?

Really? I don't think so. This is a rigid interpretation that all epistemology emanates from scientific investigation. There is room for supplemental interpretations and a genuine humbleness that all epistemology is not a function of scientific investigations.

"But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. Consider physics: Newtonian models were crude approximations of the truth (wrong at the atomic level, but still useful). A hundred years ago, statistically based quantum mechanics offered a better picture — but quantum mechanics is yet another model, and as such it, too, is flawed, no doubt a caricature of a more complex underlying reality. The reason physics has drifted into theoretical speculation about n-dimensional grand unified models over the past few decades (the "beautiful story" phase of a discipline starved of data) is that we don't know how to run the experiments that would falsify the hypotheses — the energies are too high, the accelerators too expensive, and so on."

This Popperian conclusion "...we don't know how to run the experiments that would falsify the hypotheses" is shallow in perspective in that there is the underlying assumption that all epistemology is scientific based. It futhermore excludes the possibility that ultimate universal epistemology [via scientific methodology] may be impossible of human quantification and comprhension...a species liability where the physical brain is inadequate or unable to design and execute instruments that can quantify physical events.

"The End of Theory: The Data Deluge Makes the Scientific Method Obsolete"

by

Chris Anderson

June 24th, 2008

Wired Magazine

"All models are wrong, but some are useful."

So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don't have to settle for wrong models. Indeed, they don't have to settle for models at all.

Sixty years ago, digital computers made information readable. Twenty years ago, the Internet made it reachable. Ten years ago, the first search engine crawlers made it a single database. Now Google and like-minded companies are sifting through the most measured age in history, treating this massive corpus as a laboratory of the human condition. They are the children of the Petabyte Age.

The Petabyte Age is different because more is different. Kilobytes were stored on floppy disks. Megabytes were stored on hard disks. Terabytes were stored in disk arrays. Petabytes are stored in the cloud. As we moved along that progression, we went from the folder analogy to the file cabinet analogy to the library analogy to — well, at petabytes we ran out of organizational analogies.

At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics. It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality. It forces us to view data mathematically first and establish a context for it later. For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn't pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right.

Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required. That's why Google can translate languages without actually "knowing" them (given equal corpus data, Google can translate Klingon into Farsi as easily as it can translate French into German). And why it can match ads to content without any knowledge or assumptions about the ads or the content.

Speaking at the O'Reilly Emerging Technology Conference this past March, Peter Norvig, Google's research director, offered an update to George Box's maxim: "All models are wrong, and increasingly you can succeed without them."

This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear. Out with every theory of human behavior, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.

The big target here isn't advertising, though. It's science. The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists. The models are then tested, and experiments confirm or falsify theoretical models of how the world works. This is the way science has worked for hundreds of years.

Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.

But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. Consider physics: Newtonian models were crude approximations of the truth (wrong at the atomic level, but still useful). A hundred years ago, statistically based quantum mechanics offered a better picture — but quantum mechanics is yet another model, and as such it, too, is flawed, no doubt a caricature of a more complex underlying reality. The reason physics has drifted into theoretical speculation about n-dimensional grand unified models over the past few decades (the "beautiful story" phase of a discipline starved of data) is that we don't know how to run the experiments that would falsify the hypotheses — the energies are too high, the accelerators too expensive, and so on.

Now biology is heading in the same direction. The models we were taught in school about "dominant" and "recessive" genes steering a strictly Mendelian process have turned out to be an even greater simplification of reality than Newton's laws. The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.

In short, the more we learn about biology, the further we find ourselves from a model that can explain it.

There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

The best practical example of this is the shotgun gene sequencing by J. Craig Venter. Enabled by high-speed sequencers and supercomputers that statistically analyze the data they produce, Venter went from sequencing individual organisms to sequencing entire ecosystems. In 2003, he started sequencing much of the ocean, retracing the voyage of Captain Cook. And in 2005 he started sequencing the air. In the process, he discovered thousands of previously unknown species of bacteria and other life-forms.

If the words "discover a new species" call to mind Darwin and drawings of finches, you may be stuck in the old way of doing science. Venter can tell you almost nothing about the species he found. He doesn't know what they look like, how they live, or much of anything else about their morphology. He doesn't even have their entire genome. All he has is a statistical blip — a unique sequence that, being unlike any other sequence in the database, must represent a new species.

This sequence may correlate with other sequences that resemble those of species we do know more about. In that case, Venter can make some guesses about the animals — that they convert sunlight into energy in a particular way, or that they descended from a common ancestor. But besides that, he has no better model of this species than Google has of your MySpace page. It's just data. By analyzing it with Google-quality computing resources, though, Venter has advanced biology more than anyone else of his generation.

This kind of thinking is poised to go mainstream. In February, the National Science Foundation announced the Cluster Exploratory, a program that funds research designed to run on a large-scale distributed computing platform developed by Google and IBM in conjunction with six pilot universities. The cluster will consist of 1,600 processors, several terabytes of memory, and hundreds of terabytes of storage, along with the software, including Google File System, IBM's Tivoli, and an open source version of Google's MapReduce. Early CluE projects will include simulations of the brain and the nervous system and other biological research that lies somewhere between wetware and software.

Learning to use a "computer" of this scale may be challenging. But the opportunity is great: The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.

There's no reason to cling to our old ways. It's time to ask: What can science learn from Google?

Sound science/philosophy...a product of sound thinking

Introduction to the Scientific Method:

The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.

Recognizing that personal and cultural beliefs influence both our perceptions and our interpretations of natural phenomena, we aim through the use of standard procedures and criteria to minimize those influences when developing a theory. As a famous scientist once said, "Smart people (like smart lawyers) can come up with very good explanations for mistaken points of view." In summary, the scientific method attempts to minimize the influence of bias or prejudice in the experimenter when testing an hypothesis or a theory.

The scientific method has four steps:

1. Observation and description of a phenomenon or group of phenomena.

2. Formulation of an hypothesis to explain the phenomena. In physics, the hypothesis often takes the form of a causal mechanism or a mathematical relation.

3. Use of the hypothesis to predict the existence of other phenomena, or to predict quantitatively the results of new observations.

4. Performance of experimental tests of the predictions by several independent experimenters and properly performed experiments.

If the experiments bear out the hypothesis it may come to be regarded as a theory or law of nature (more on the concepts of hypothesis, model, theory and law below). If the experiments do not bear out the hypothesis, it must be rejected or modified. What is key in the description of the scientific method just given is the predictive power (the ability to get more out of the theory than you put in; see Barrow, 1991) of the hypothesis or theory, as tested by experiment. It is often said in science that theories can never be proved, only disproved. There is always the possibility that a new observation or a new experiment will conflict with a long-standing theory.

Testing hypotheses:

As just stated, experimental tests may lead either to the confirmation of the hypothesis, or to the ruling out of the hypothesis. The scientific method requires that an hypothesis be ruled out or modified if its predictions are clearly and repeatedly incompatible with experimental tests. Further, no matter how elegant a theory is, its predictions must agree with experimental results if we are to believe that it is a valid description of nature. In physics, as in every experimental science, "experiment is supreme" and experimental verification of hypothetical predictions is absolutely necessary. Experiments may test the theory directly (for example, the observation of a new particle) or may test for consequences derived from the theory using mathematics and logic (the rate of a radioactive decay process requiring the existence of the new particle). Note that the necessity of experiment also implies that a theory must be testable. Theories which cannot be tested, because, for instance, they have no observable ramifications (such as, a particle whose characteristics make it unobservable), do not qualify as scientific theories.

If the predictions of a long-standing theory are found to be in disagreement with new experimental results, the theory may be discarded as a description of reality, but it may continue to be applicable within a limited range of measurable parameters. For example, the laws of classical mechanics (Newton's Laws) are valid only when the velocities of interest are much smaller than the speed of light (that is, in algebraic form, when v/c <<>> 10-8 m). A description which is valid at all length scales is given by the equations of quantum mechanics.

We are all familiar with theories which had to be discarded in the face of experimental evidence. In the field of astronomy, the earth-centered description of the planetary orbits was overthrown by the Copernican system, in which the sun was placed at the center of a series of concentric, circular planetary orbits. Later, this theory was modified, as measurements of the planets motions were found to be compatible with elliptical, not circular, orbits, and still later planetary motion was found to be derivable from Newton's laws.

Error in experiments have several sources. First, there is error intrinsic to instruments of measurement. Because this type of error has equal probability of producing a measurement higher or lower numerically than the "true" value, it is called random error. Second, there is non-random or systematic error, due to factors which bias the result in one direction. No measurement, and therefore no experiment, can be perfectly precise. At the same time, in science we have standard ways of estimating and in some cases reducing errors. Thus it is important to determine the accuracy of a particular measurement and, when stating quantitative results, to quote the measurement error. A measurement without a quoted error is meaningless. The comparison between experiment and theory is made within the context of experimental errors. Scientists ask, how many standard deviations are the results from the theoretical prediction? Have all sources of systematic and random errors been properly estimated? This is discussed in more detail in the appendix on Error Analysis and in Statistics Lab 1.

Common Mistakes in Applying the Scientific Method:

As stated earlier, the scientific method attempts to minimize the influence of the scientist's bias on the outcome of an experiment. That is, when testing an hypothesis or a theory, the scientist may have a preference for one outcome or another, and it is important that this preference not bias the results or their interpretation. The most fundamental error is to mistake the hypothesis for an explanation of a phenomenon, without performing experimental tests. Sometimes "common sense" and "logic" tempt us into believing that no test is needed. There are numerous examples of this, dating from the Greek philosophers to the present day.

Another common mistake is to ignore or rule out data which do not support the hypothesis. Ideally, the experimenter is open to the possibility that the hypothesis is correct or incorrect. Sometimes, however, a scientist may have a strong belief that the hypothesis is true (or false), or feels internal or external pressure to get a specific result. In that case, there may be a psychological tendency to find "something wrong", such as systematic effects, with data which do not support the scientist's expectations, while data which do agree with those expectations may not be checked as carefully. The lesson is that all data must be handled in the same way.

Another common mistake arises from the failure to estimate quantitatively systematic errors (and all errors). There are many examples of discoveries which were missed by experimenters whose data contained a new phenomenon, but who explained it away as a systematic background. Conversely, there are many examples of alleged "new discoveries" which later proved to be due to systematic errors not accounted for by the "discoverers."

In a field where there is active experimentation and open communication among members of the scientific community, the biases of individuals or groups may cancel out, because experimental tests are repeated by different scientists who may have different biases. In addition, different types of experimental setups have different sources of systematic errors. Over a period spanning a variety of experimental tests (usually at least several years), a consensus develops in the community as to which experimental results have stood the test of time.

Hypotheses, Models, Theories and Laws:

In physics and other science disciplines, the words "hypothesis," "model," "theory" and "law" have different connotations in relation to the stage of acceptance or knowledge about a group of phenomena.

An hypothesis is a limited statement regarding cause and effect in specific situations; it also refers to our state of knowledge before experimental work has been performed and perhaps even before new phenomena have been predicted. To take an example from daily life, suppose you discover that your car will not start. You may say, "My car does not start because the battery is low." This is your first hypothesis. You may then check whether the lights were left on, or if the engine makes a particular sound when you turn the ignition key. You might actually check the voltage across the terminals of the battery. If you discover that the battery is not low, you might attempt another hypothesis ("The starter is broken"; "This is really not my car.")

The word model is reserved for situations when it is known that the hypothesis has at least limited validity. A often-cited example of this is the Bohr model of the atom, in which, in an analogy to the solar system, the electrons are described has moving in circular orbits around the nucleus. This is not an accurate depiction of what an atom "looks like," but the model succeeds in mathematically representing the energies (but not the correct angular momenta) of the quantum states of the electron in the simplest case, the hydrogen atom. Another example is Hook's Law (which should be called Hook's principle, or Hook's model), which states that the force exerted by a mass attached to a spring is proportional to the amount the spring is stretched. We know that this principle is only valid for small amounts of stretching. The "law" fails when the spring is stretched beyond its elastic limit (it can break). This principle, however, leads to the prediction of simple harmonic motion, and, as a model of the behavior of a spring, has been versatile in an extremely broad range of applications.

A scientific theory or law represents an hypothesis, or a group of related hypotheses, which has been confirmed through repeated experimental tests. Theories in physics are often formulated in terms of a few concepts and equations, which are identified with "laws of nature," suggesting their universal applicability. Accepted scientific theories and laws become part of our understanding of the universe and the basis for exploring less well-understood areas of knowledge. Theories are not easily discarded; new discoveries are first assumed to fit into the existing theoretical framework. It is only when, after repeated experimental tests, the new phenomenon cannot be accommodated that scientists seriously question the theory and attempt to modify it. The validity that we attach to scientific theories as representing realities of the physical world is to be contrasted with the facile invalidation implied by the expression, "It's only a theory." For example, it is unlikely that a person will step off a tall building on the assumption that they will not fall, because "Gravity is only a theory."

Changes in scientific thought and theories occur, of course, sometimes revolutionizing our view of the world (Kuhn, 1962). Again, the key force for change is the scientific method, and its emphasis on experiment.

Are there circumstances in which the Scientific Method is not applicable?:

While the scientific method is necessary in developing scientific knowledge, it is also useful in everyday problem-solving. What do you do when your telephone doesn't work? Is the problem in the hand set, the cabling inside your house, the hookup outside, or in the workings of the phone company? The process you might go through to solve this problem could involve scientific thinking, and the results might contradict your initial expectations.

Like any good scientist, you may question the range of situations (outside of science) in which the scientific method may be applied. From what has been stated above, we determine that the scientific method works best in situations where one can isolate the phenomenon of interest, by eliminating or accounting for extraneous factors, and where one can repeatedly test the system under study after making limited, controlled changes in it.

There are, of course, circumstances when one cannot isolate the phenomena or when one cannot repeat the measurement over and over again. In such cases the results may depend in part on the history of a situation. This often occurs in social interactions between people. For example, when a lawyer makes arguments in front of a jury in court, she or he cannot try other approaches by repeating the trial over and over again in front of the same jury. In a new trial, the jury composition will be different. Even the same jury hearing a new set of arguments cannot be expected to forget what they heard before.

Conclusion:

The scientific method is intricately associated with science, the process of human inquiry that pervades the modern era on many levels. While the method appears simple and logical in description, there is perhaps no more complex question than that of knowing how we come to know things. In this introduction, we have emphasized that the scientific method distinguishes science from other forms of explanation because of its requirement of systematic experimentation. We have also tried to point out some of the criteria and practices developed by scientists to reduce the influence of individual or social bias on scientific findings. Further investigations of the scientific method and other aspects of scientific practice may be found in the references listed below.

References:

1. Wilson, E. Bright. An Introduction to Scientific Research (McGraw-Hill, 1952).

2. Kuhn, Thomas. The Structure of Scientific Revolutions (Univ. of Chicago Press, 1962).

3. Barrow, John. Theories of Everything (Oxford Univ. Press, 1991).

c. University of Rochester


INFORMAL FALLACIES

Number One

Number Two

Number Three


And some common sense from Carl Sagan's "Baloney Detection Kit"