Testing quotes

From Motley Bytes Wiki
Jump to: navigation, search

Jim_Baggott

The Testability Principle

“The Testability Principle. The principal requirement of a scientific theory is that it should in some way be testable through reference to existing or new facts about empirical reality. The test exposes the veracity or falsity of the theory, but there is a caveat. The working stuff of theories is itself abstract and metaphysical. Getting this stuff to apply to the facts or a test situation typically requires wrapping the abstract concepts in a blanket of auxiliary assumptions; some explicitly stated, many taken as read. This means that a test is rarely decisive. When a test shows that a theory is false, the theory is not necessarily abandoned. It may simply mean that one or more of the auxiliary assumptions are wrong.”
  • Source: Baggott, Jim. Farewell to Reality: How Modern Physics Has Betrayed the Search for Scientific Truth (p. 20). Pegasus Books. New York, NY.

Robert_N_Britcher

Limits of Testing

“The limits of software are never more obvious than in its testing. Software systems are beyond discrete and statistical analysis. A single incorrect state - out of two to the trillionth states - can down a rocket: To find such a fault in a software system would require a hundred generations of programmers - but after the first failure is found and fixed, all bets are off; they wouldn't be testing the same system anymore. We can design and build fail-safe systems, but can we know we did?”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 56.

Sources of Failures

“I never ran a test that didn't find at least one failure - not only is the software impure, but so are the requirements specifications. And every time we fix the software, there is a 10 percent chance - according to my anecdotal studies - we'll inject another fault. Those are the hardest to find: the faults that result from trying to get rid of them.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 56.

Testing Large Systems

“Many programmers do an adequate job of testing the logic of a single module. But testing an entire system is another matter. Once a module is absorbed into the mass of programs, running as part of the real world, the nature of testing changes.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 57.

Scientific Method and Testing

“The scientific method is not part of their curriculum. The men and women who are called upon to test computer systems know little, if anything, about running a simple experiment.”
  • Source:Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 58.

Errors in Tests and Simulators

“The simulator software had to be tested as thoroughly as the software it was built to test. A fault in the simulator would cause the worst kind of problem, what computer analysts call a Byzantine failure. Byzantine failures are like traitors in a family. There is always a risk this will happen.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 61.

Failures Disappointing

“... discovering a failure is disappointing, and then there is pressure from above not to find them. Everyone wants the software out the door.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 61.

When to Stop Testing

“Two of the great questions in developing computer systems are (1) When do we stop testing? and (2) At any point during testing, how far along are we? The second question cannot be answered until the first one has been answered. The first question can be answered by fiat.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 61.

Failure Discovery Rate

“A critical parameter in the development of any system is the discovery rate of software failures, especially during system testing.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 62.

Verification Percent

“Over 70 percent of the technical budget was spent on verification activities such as inspections and testing; only 30 percent was spent designing and writing code.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 117.

Software Testing Completion Criteria

“Software is never completely fit. ... Testing should simply continue until all of the performance criteria have been met, including the failure rate.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 157.

Acceptance Testing

“The very existence of an acceptance test ensures that, sooner or later, there will be pressure not to find failures, to reconcile test scenarios and the system they exercise so that the system will 'pass'. Remove this concept and testers would pursue the discovery of failures with a vengeance...”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 157.

Requirements Testing

“Requirements testing is a subtle way of treating a nonlinear, unpredictable system as if it were a discrete symbol like a number or a word. The potential and the actual become muddled. Requirements are natural language statements, peppered in places with arithmetic. These are discrete and usually quite vague representations of the system as it might become. But the truth is that two teams writing requirements for the same system would describe them differently, and neither would get it quite right, simply because 'right' cannot be known until the system is built. ... Requirements testing is a little like trying to reduce crime in England by running a test for each line in the Magna Carta. In practice, rhetoric replaces the scientific method. As with mathematical proofs, the quest for 'true or not false' soon turns into a party - a not very cordial party - marked by endless debate over whether this sentence or that sentence has been verified by this moment of behavior or that.”
  • Source: Britcher, Robert N. (1999). The Limits of Software: People, Projects, and Perspectives. Reading, MA: Addison Wesley. ISBN 0-201-43323-0. Page 157.

Fritjof_Capra

Quantum Theory Acceptance

"The mathematical framework of quantum theory has passed countless successful tests and is now universally accepted as a consistent and accurate description of all atomic phenomena. The verbal interpretation, on the other hand – i.e., the metaphysics of quantum theory – is on far less solid ground. In fact, in more than forty years physicists have not been able to provide a clear metaphysical model."
  • Source: Fritjof Capra, In The Tao of Physics (1975), 132.

W_Edwards_Deming

Quality and Inspection

"Quality comes not from inspection, but from improvement of the production process."
  • Source: W. Edwards Deming (1982) Out Of The Crisis. The MIT Press. (2e ed. 1986) p. 29.

Mass Inspection and Quality

"We cannot rely on mass inspection to improve quality, though there are times when 100 percent inspection is necessary. As Harold S. Dodge said many years ago, 'You cannot inspect quality into a product.' The quality is there or it isn't by the time it's inspected."
  • Source: W. Edwards Deming (1982) Out Of The Crisis. The MIT Press. (2e ed. 1986) p. 139.

David_Deutsch

Theory Rejection

“The overwhelming majority of theories are rejected because they contain bad explanations, not because they fail experimental tests.”

Edsger_Wybe_Dijkstra

Testing Bug Presence

“Testing shows the presence, not the absence of bugs.“
  • Source: Dijkstra (1969) J.N. Buxton and B. Randell, eds, Software Engineering Techniques, April 1970, p. 16. Report on a conference sponsored by the NATO Science Committee, Rome, Italy, 27–31 October 1969.

Michael_Faraday

Laws of Nature

“The laws of nature, as we understand them, are the foundation of our knowledge in natural things. So much as we know of them has been developed by the successive energies of the highest intellects, exerted through many ages. After a most rigid and scrutinizing examination upon principle and trial, a definite expression has been given to them; they have become, as it were, our belief or trust. From day to day we still examine and test our expressions of them. We have no interest in their retention if erroneous. On the contrary, the greatest discovery a man could make would be to prove that one of these accepted laws was erroneous, and his greatest honour would be the discovery.”
  • Source: Book/Author=Michael Faraday; Title=“Experimental Researches in Chemistry and Physics”; Publication/Date=1859; Page=469.

John_Gribbin

Scientific Method

“The Big Idea that had been developed in the seventeenth century ... is now known as the scientific method. It says that the way to proceed when investigating how the world works is to first carry out experiments and/or make observations of the natural world. Then, develop hypotheses to explain these observations, and (crucially) use the hypothesis to make predictions about the future outcome of future experiments and/or observations. After comparing the results of those new observations with the predictions of the hypotheses, discard those hypotheses which make false predictions, and retain (at least, for the time being) any hypothesis that makes accurate predictions, elevating it to the status of a theory. Note that a theory can never be proved right. The best that can be said is that it has passed all the tests applied so far.”
  • Source: John R. Gribbin. In The Fellowship: the Story of a Revolution (2005), 275.

Stephen_Hawking

Correspondence to Reality

“I don't demand that a theory corresponds to reality because I don't know what it is. Reality is not a quality you can test with litmus paper. All I'm concerned with is that the theory should predict the results of experiment.”
  • Source: During a debate with Roger Penrose in 1994 at the Isaac Newton Institute for Mathematical Sciences at the University of Cambridge, transcribed in The Nature of Space and Time (1996) by Stephen Hawking and Roger Penrose, Princeton, NJ: Princeton Umiversity Press, p. 121.

Hermann_von_Helmholtz

Movement and Perception

“Each movement we make by which we alter the appearance of objects should be thought of as an experiment designed to test whether we have understood correctly the invariant relations of the phenomena before us.”
  • Source: Helmholtz (1878) The Facts of Perception

Edwin_Hubble

Scientific Guesses

“[A scientist] naturally and inevitably ... mulls over the data and guesses at a solution. [He proceeds to] testing of the guess by new data — predicting the consequences of the guess and then dispassionately inquiring whether or not the predictions are verified.”
  • Source: Edwin Powell Hubble, From manuscript on Francis Bacon as a scientist (1942), Edwin Hubble collection, Box 2, Huntington Library, San Marino, California. As cited by Norriss S. Hetherington in 'Philosophical Values and Observation in Edwin Hubble's Choice of a Model of the Universe', Historical Studies in the Physical Sciences (1982), 13, No. 1, 42.

Brooks_Jackson and Kathleen_Hall_Jamieson

Psychological Experiments

“Psychological experiments have shown ... that humans tend to seek out even weak evidence to support their existing beliefs, and to ignore evidence that undercuts those beliefs. In the process, we apply stringent tests to evidence we don't want to hear, while letting slide uncritically into our minds any information that suits our needs.”
  • Source: Brooks Jackson, As co-author with Kathleen Hall Jamieson, in unSpun: Finding Facts in a World of Disinformation (2007), 69-70.

David_Kipping

Nature Computation

“It is important to distinguish between testing whether the universe is built upon the principles of computation but is not a simulation versus testing where the universe is indeed a simulation on someone's computer. Those are different to each other. It is quite possible that the laws of computation are somehow embedded within nature and yet everything we see around us is not a simulation.”
  • Source: David Kipping from the YouTube “Why You're Probably Not a Simulation”, posted 2020-09-04 in the channel ‘Cool Worlds’. Video offset 11:15. Online at https://youtu.be/HA5YuwvJkpQ (2025).

Alan_Perlis

Iterative Simulation Design

“1. A software system can best be designed if the testing is interlaced with the designing instead of being used after the design.
2. A simulation which matches the requirements contains the control which organizes the design of the system.
3. Through successive repetitions of this process of interlaced testing and design the model ultimately becomes the software system itself.
I think that it is the key of the approach that has been suggested, that there is no such question as testing things after the fact with simulation models, but that in effect the testing and the replacement of simulations with modules that are deeper and more detailed goes on with the simulation model controlling, as it were, the place and order in which these things are done.”

Steven_Pinker

Neurons to Consciousness

“None of us has a brain that can actually put us in touch with reality unaided with truth. I've just shown you a bunch of gadgetry that responds to things that are essential to our survival, things that prevent us from falling off cliffs and bumping into trees. But they don't actually correspond to any clear description of the world as it is. So we have, for example, circuitry that responds to the edges rather than the physical distribution of light, because edges are important, that's where objects leave off and their backgrounds begin. Moreover, they can be fooled, as I did with each one of you. That since our neural circuitry does consist of a mechanism, not a pipeline to the truth or a window onto objective reality. That is naïve realism we know is false because there are illusions. You can't just trust your senses. That means we need some way of augmenting the faculties that nature gave us to make our beliefs as accurate as possible. Which of course is why the early modern philosophers made such a big deal of visual illusions. They said you can't just open your eyes and see the world as it is. Look at all these illusions. So what we do, what gives some of us confidence that our beliefs are objectively true is that they are part of an institutional infrastructure of testing and criticism and logic and determining which hypotheses withstand criticism and attempts at refutation. Devising exotic arrangements of reality to reflect the way reality is put together, namely experiments. Thanks to this huge network of minds, each one of them making up for deficits of all the others, we like to think that we have a collective grasp on truth that none of us individually can have. But of course, it's always fallible. The religious beliefs I think were actually pretty reasonable hypotheses in earlier states of our collective knowledge. Even Richard Dawkins said that before Darwin he probably would have been a creationist because the cosmic engineer was the only available explanation for the obvious adaptive complexity of living things. I would add that prior to the advent of modern neuroscience I probably would have believed in souls because there was no better explanation for phenomena such as that people respond intelligently to their environments. When you dream some part of you is up and about the world while your body is in bed the whole time. That seems like there must be some part of you that can leave your body. When a person suddenly dies some kind of animating force no longer seems to be animating the body that was there just a few minutes ago. So the soul hypothesis is not a stupid or irrational belief, nor is creationism if it's the best that you have. Now, of course, we can do better. But doing better in part depends not just on this cumulative weight of evidence and argument. But because few of us can actually reconstruct the scientific observations that justify the theory of natural selection or the theory of neural networks, all of us, at least a little bit, take it on faith in the following sense; that there is a community that you trust enough because when they have been challenged to defend their beliefs they can connect them to reality. They have shown that it isn't magic. That they have given reason to trust them because they've convinced you that they're the mores of their culture, of their scientific culture, designed so that they will find true things and weed out false things. If you don't have a degree of solidarity or trust, if you think that scientists are all in cahoots with big corporations or are trying to tear down all faith and values and meaning and you simply refuse to follow through the very long chain of deduction that connects the claims to the experiments, then you'll naturally reject it just because it's not your tribe. For you and me scientists are our tribe and we're apt to give them some credibility, indeed some credulousness, knowing that if ever challenged it would be repaid. But unless you trace that through it can be all too easy to reject the conclusions of science, and indeed a number of studies have shown that contrary to the common belief that people believe in falsehoods, such as that vaccines cause autism or that humans were created five thousand years ago or that the burning of fossil fuels has no effect on global temperatures, it turns out that people with those beliefs that we have every reason to believe are false are no more scientifically naïve on average than people that have beliefs that are scientifically justified. You give a test on evolution to a bunch of people who believe in evolution, give it to a bunch of creationists, there's not much difference in how well they score. In fact, often they are equal. Likewise, the typical grasp of climate science among someone who signs on to the scientific consensus is not much better than the adamant deniers. That kind of dashes the hope that simply by increasing the quality of science education you will lead people to more accurate beliefs. There, in addition, has to be a dissociation of who you think are the good guys and who are the bad guys from who has the best claims to truth. That's what it depends on.”

Keywords: Accuracy_quotes Adament_quotes Adaptability_quotes Addition_quotes Advent_quotes Aid_quotes Animate_quotes Argue_quotes Arrangement_quotes Attempt_quotes Augment_quotes Autism_quotes Availability_quotes Background_quotes Bad_quotes Bed_quotes Belief_quotes Best_quotes Big_quotes Body_quotes Brain_quotes Bump_quotes Bunch_quotes Burn_quotes Cahoot_quotes Chain_quotes Challenge_quotes Circuit_quotes Claim_quotes Clear_quotes Cliff_quotes Climate_quotes Collective_quotes Complexity_quotes Common_quotes Community_quotes Conclusion_quotes Confidence_quotes Connection_quotes Consensus_quotes Consist_quotes Contrary_quotes Convince_quotes Cooperation_quotes Corporation_quotes Correspondence_quotes Cosmic_quotes Creationism_quotes Credibility_quotes Criticism_quotes Culture_quotes Cumulative_quotes Dash_quotes Deal_quotes Death_quotes Deception_quotes Deduction_quotes Defence_quotes Deficit_quotes Degree_quotes Deny_quotes Dependence_quotes Description_quotes Design_quotes Determine_quotes Devise_quotes Differences_quotes Dissociation_quotes Distribution_quotes Dreaming_quotes Early_quotes Easy_quotes Edge_quotes Education_quotes Effect_quotes Engineer_quotes Environment_quotes Essential_quotes Evidence_quotes Evolution_quotes Exotic_quotes Experiment_quotes Explanation_quotes Eye_quotes Faculty_quotes Faith_quotes Fall_quotes Fallible_quotes False_quotes Few_quotes Find_quotes Follow_quotes Fool_quotes Force_quotes Fossil_quotes Fuel_quotes Gadget_quotes Global_quotes Good_quotes Grasp_quotes Guy_quotes Huge_quotes Hypothesis_quotes Illusion_quotes Importance_quotes Increasing_quotes Individual_quotes Infrastructure_quotes Institution_quotes Intelligence_quotes Irrational_quotes Justification_quotes Knowledge_quotes Leave_quotes Life_quotes Light_quotes Logic_quotes Long_quotes Magic_quotes Meaning_quotes Mechanism_quotes Mind_quotes Modern_quotes Moral_quotes Naïve_quotes Nature_quotes Natural_Selection_quotes Network_quotes Neural_quotes Neuroscience_quotes Object_quotes Objectivity_quotes Observation_quotes Obvious_quotes Part_quotes People_quotes Perception_quotes Phenomena_quotes Philosophy_quotes Physical_quotes Pipeline_quotes Possibility_quotes Prehistoric_quotes Prevent_quotes Reality_quotes Reason_quotes Reconstruct_quotes Reflection_quotes Refuse_quotes Refutation_quotes Rejection_quotes Religion_quotes Repay_quotes Respond_quotes Science_quotes Scientist_quotes Score_quotes Seem_quotes Selection_quotes Sense_quotes Show_quotes Sight_quotes Solidarity_quotes Soul_quotes State_quotes Study_quotes Stupidity_quotes Survival_quotes Tear_quotes Temperature_quotes Testing_quotes Theory_quotes Thinking_quotes Time_quotes Touch_quotes Trace_quotes Tree_quotes Tribal_quotes Trust_quotes Truth_quotes Try_quotes Typical_quotes Unaided_quotes Vaccine_quotes Value_quotes Visual_quotes Weed_quotes Weight_quotes Whole_quotes Window_quotes Withstand_quotes World_quotes

Science

“In a large society with writing and institutionalized science, the cost of an exponential number of tests is repaid by the benefit of the resulting laws to a large number of people. That is why taxpayers are willing to fund scientific research. But for the provincial interests of a single individual or even a small band, good science isn’t worth the trouble.”
  • Source: Steven Pinker (1997). How the Mind Works. New York, NY: W. W. Norton. ISBN 978-0-393-06973-0; eISBN 978-0-393-06973-0. Chapter 5 Section: Ecological Intelligence.

Robert_M_Pirsig

An Experiment is Never a Failure

“An experiment is never a failure solely because it fails to achieve predicted results. An experiment is a failure only when it also fails adequately to test the hypothesis in question, when the data it produces don’t prove anything one way or another.”
  • Source: Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values (1974), 102.

Traditional Scientific Method

“Traditional scientific method has always been at the very best, 20-20 hindsight. It's good for seeing where you've been. It's good for testing the truth of what you think you know, but it can't tell you where you ought to go.”
  • Source: Robert Pirsig, Zen and the Art of Motorcycle Maintenance (1974), 251.

Karl_Popper

Confirmation Easy

“It is easy to obtain confirmations, or verifications, for nearly every theory — if we look for confirmations. Confirmations should count only if they are the result of risky predictions... A theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice. Every genuine test of a theory is an attempt to falsify it, or refute it.”
  • Source: Karl Raimund Popper, Conjectures and Refutations: The Growth of Scientific Knowledge (1963), 36.

Method of Science

“The method of science depends on our attempts to describe the world with simple theories: theories that are complex may become untestable, even if they happen to be true. Science may be described as the art of systematic over-simplification — the art of discerning what we may with advantage omit.”
  • Source: Karl Raimund Popper, Karl Raimund Popper and William Warren Bartley (ed.), The Open Universe: an Argument for Indeterminism (1991), 44. by Karl Raimund Popper, William Warren Bartley - Science - 1991

Considerations

“These considerations led me in the winter of 1919-20 to conclusions which I may now reformulate as follows.
1. Confirmation is easy.
2. Only risky predictions should count.
3. A good theory is a prohibition.
4. A theory which is not refutable is not scientific.
5. A genuine test is an attempt to falsify a theory.
6. Confirming evidence should not count unless it attempted to falsify.
7. Some genuine testable theories, when found to be false, are still upheld by their admirers.”

Epistemology

“In accordance with my proposal made above, epistemology, or the logic of scientific discovery, should be identified with the theory of scientific method. The theory of method, in so far as it goes beyond the purely logical analysis of the relations between scientific statements, is concerned with the choice of methods — with decisions about the way in which scientific statements are to be dealt with. These decisions will of course depend in their turn upon the aim which we choose from among a number of possible aims. The decision here proposed for laying down suitable rules for what I call the ‘empirical method’ is closely connected with my criterion of demarcation: I propose to adopt such rules as will ensure the testability of scientific statements; which is to say, their falsifiability.”
  • Source: Chapter 2. Popper, K. R. (2005). The Logic of Scientific Discovery | 1935 (1935/Epub ed.). New York, NY: Routledge.

Alfred_Russel_Wallace

Truth Proof

“There is no more convincing proof of the truth of a comprehensive theory than its power of absorbing and finding a place for new facts, and its capability of interpreting phenomena which had been previously looked upon as unaccountable anomalies. It is thus that the law of universal gravitation and the undulatory theory of light have become established and universally accepted by men of science. Fact after fact has been brought forward as being apparently inconsistent with them, and one after another these very facts have been shown to be the consequences of the laws they were at first supposed to disprove. A false theory will never stand this test. Advancing knowledge brings to light whole groups of facts which it cannot deal with, and its advocates steadily decrease in numbers, notwithstanding the ability and scientific skill with which it may have been supported.”
  • Source: Alfred Russel Wallace, From a review of four books on the subject 'Mimicry, and Other Protective Resemblances Among Animals', in The Westminster Review (Jul 1867), 88, 1. Wallace is identified as the author within the article as reprinted in William Beebe, The Book of Naturalists: An Anthology of the Best Natural History (1988), 108.

Rupert_Sheldrake

Multiverse

“This multiverse theory is the ultimate violation of Occam’s Razor, the philosophical principle that ‘entities must not be multiplied beyond necessity,' or in other words, that we should make as few assumptions as possible. It also has the major disadvantage of being untestable. And it does not even succeed in getting rid of God. An infinite God could be the God of an infinite number of universes.”
  • Source: Rupert Sheldrake. The book “Science Set Free: 10 Paths to New Discovery”, Introduction, page 12. Published in 2012 by Crown Publishing, New York, NY. Kindle Edition. ISBN: 978-0-7704-3671-1.

Eugene_Wigner

No Definite Evidence

“[F. Werner, while a student in Princeton,] came to me and expressed his bewilderment with the fact that we make a rather narrow selection when choosing the data on which we test our theories. ‘How do we know that, if we made a theory which focuses its attention on phenomena we disregard and disregards some of the phenomena now commanding our attention, that we could not build another theory which has little in common with the present one but which, nevertheless, explains just as many phenomena as the present theory?' It has to be admitted that we have no definite evidence that there is no such theory.”
  • Source: Eugene Paul Wigner, In 'The Unreasonable Effectiveness of Mathematics in the Natural Sciences,' Communications in Pure and Applied Mathematics (Feb 1960), 13, No. 1 (February 1960). Collected in Eugene Paul Wigner, A.S. Wightman (ed.), Jagdish Mehra (ed.), The Collected Works of Eugene Paul Wigner (1955), Vol. 6, 535.