In 1932, with memories of his 1927 solo flight across the atlantic Ocean still fresh, Charles Lindbergh was one of the best known and most highly respected people in the world. His wife, Anne, was the daughter of a partner in the J. P. Morgan bank who had also served as ambassador to Mexico. At approximately 7:00 p.m., on the evening of Tuesday, March 1, 1932, the Lindberghs put their 20-month-old son, Charles, to bed in his upstairs room at their country estate near Hopewell, New Jersey. Construction was not quite completed on the house, and usually the family only stayed there on weekends. Because Charles had had a cold, however, they had stayed longer. Also in the house that evening were three servants: a butler, a cook, and a nursemaid. At approximately 10:00 p.m., the nursemaid checked on the baby and found the crib empty. One of the bedroom windows was open, and there was a small, white envelope on the sill. A quick search inside and outside the house turned up no sign of the child. The local police responded quickly to a call, and a further search revealed a custom-made collapsible ladder and a chisel in the yard.

One of the early casualties in the investigation was the crime scene. The first detectives to arrive did not take accurate measurements or make plaster casts of footprints and tire tracks that they found. In addition, as more police, reporters, and sightseers arrived, these prints and tracks were obliterated.

Under england's queen elizabeth i (1533–1603) tensions with Spain, which had once been an ally, increased over religion, trade, and other issues. From the mid-1580s there was an undeclared naval war in European waters, as well as in Spain's colonies throughout the world. To resolve the rivalry, King Philip II of Spain (1527–98) decided to build a great fleet (Armada) for an invasion of England.

The method of the attack was fairly well known: It would be carried out by large wooden sailing ships. Since the beginning of the sixteenth century there had been such ships, carrying sizable numbers of cannons and troops anywhere on earth where there was water and wind, which was approximately 70 percent of the planet's surface. This capability to send men and equipment in considerable numbers over vast distances was unprecedented. For the English government, the key questions were to determine (1) what Philip would choose as the target and (2) when an attack an attack might come. The information on which to base planning to resist the Armada came in two separate channels: the experience of the English navy and the country's espionage network.

Spain had a global empire, with territory in Europe, the Americas, and the Philippines. The English navy, which was then an emerging organization, learned about Spain's galleons through direct contact in battles around the world, starting in the 1560s. Mariners, such as John Hawkins (1532–95) and Francis Drake (1540–96), found that Spanish ships were powerful, but also slow and awkward.

Analysis is as much about questions as it is about answers. it is about understanding various ways to think about a problem and realizing that not every analytic method applies to every problem. The effective analyst has a collection of approaches, techniques, and tools, along with the willingness and patience to apply them. These analytic methods have been accumulated over time and from many different sources.

Uncertainty

To deal with uncertainty, analysts seek to provide context, a sense of the bigger picture. A useful first step in understanding context is problem restatement, also known as bounding the problem, which involves making sure that all relevant and important aspects of the issue are taken into account. Virtually every issue has various aspects, such as economic, social, political, and legal; therefore, analysts need to take the various perspectives into account. Analysts also provide context by looking for abstract linkages such as patterns, relationships, and trends. Understanding such linkages involves asking such questions as whether developments are new, accelerated, or having more impact.

A specific tool for providing context is the chronology, or the listing of events in the order in which they happened. This order may well be different from the order in which reports about the events were received; thus it promotes clarity about where individual events fit into the overall pattern. Briefly summarizing the various events, to fit them into a chronology, also helps analysts to focus on the key elements and the relative value of reports.

Many of the same analytic frameworks and procedures used in national security (e.g., hypotheses, trends, relationships, and indicators) are also useful in law enforcement. In fact, some widely used analytic techniques, such as network analysis, were pioneered in law enforcement. There are also some similarities in collection based on multiple kinds of sources. Both national security and law enforcement use imagery (surveillance photos and video in police work), reports from human sources (witness statements, confidential informants), and communications intercepts (telephone taps), for example.

There are also significant differences, however. Instead of trying to anticipate and plan for developments, law enforcement generally does not act until after a crime has taken place. Given the high stakes and the pressure of time, military and political leaders often have to make decisions before the facts are clear. The judicial system can take more time, and has higher standards of proof: for a case, “preponderance of evidence” in civil cases and “beyond a reasonable doubt” in criminal ones. Court procedures, such as maintaining a clear and unadulterated chain of custody for evidence, are much more stringent than the way “evidence” is handled in the national security agencies. The consumers of intelligence are also different in law enforcement. They often include local political and police officials who need to carefully apportion scarce resources to pursue investigations and prosecutions that will succeed and have an impact. Consumers can also include operators, who use analysis to target key suspects for arrest.

The cold war was waged not only in the third world and on missile production lines, but also in quieter ways. During the waning months of 1985, American spies in the USSR were being arrested at an alarming rate, and by the following spring almost a dozen – virtually the entire roster of human sources operated by the Central Intelligence Agency (CIA) and the Federal Bureau of Investigation (FBI) in the Soviet Union – had been killed. Most were quickly executed; in some cases their fate was never determined. As time passed, a number of technical operations being conducted against the Soviets were also exposed. Obviously, something was terribly wrong.

In 1986, the CIA and FBI launched separate counterintelligence investigations to try to determine what had happened, so that corrective measures could be taken. The CIA's approach was to look at the records of the blown cases to see if there were common elements that might provide leads. Initially the hypotheses were: (1) that the problem was sloppy tradecraft by the Soviet agents; (2) revelations from the numerous spies arrested in 1985 (which had become known as “the year of the spy”); or (3) that U.S. secure communications with Moscow had been compromised.

It soon became clear that none of these hypotheses was a plausible explanation. That all of the American spies had made operational mistakes at the same time was unlikely.

As far back as there have been organized societies and records of their activities, there have been wars. To achieve victory in those conflicts, military commanders realized that it was to their advantage to have accurate information about factors such as the lay of the land, the strength and location of enemy forces, and the opposing commander's capabilities and intentions.

Looking back from the perspective of the age of the Internet and trying to understand the analytic challenges centuries ago, it is worthwhile to think for a few moments about how different things were.

Records are not always complete or accurate when looking back at the distant past. It is clear, however, that the power, range, and mobility of weapons were quite restricted. The introduction of gunpowder increased capabilities, but muskets and cannons still had limited range. Moreover, armies and fleets could move only at the speed at which a man could march or a ship sail. Providing sufficient supplies also limited the size, duration, and scope of operations.

Information – the lifeblood of intelligence – could move a bit faster than armies or fleets, at the speed of a galloping horse, although this speed was difficult to maintain over extremely long distances.

In the ancient world, the effort to obtain accurate and useful information before making a decision revealed what would turn out to be timeless problems such as how to assess risk, what constitutes reliable evidence, how to uncover and deal with deception, and how to avoid getting caught by surprise.

In recent years there has been much discussion of success and failure in intelligence and many proposals for reform. Much of the focus has been on reorganizations, such as establishing the position of Director of National Intelligence to improve coordination among the various intelligence agencies and creating the Department of Homeland Security, or more advanced technology, such as data mining.

Relatively little attention, however, has been devoted to how to improve analytic thinking in intelligence, especially by examining the historical record to see which analytic approaches have worked – or not worked. To fill this gap, Challenges in Intelligence Analysis: Lessons from 1300 BCE to the Present offers examples from the last three millennia. These examples have been drawn from around the world and a variety of disciplines, including national security, law enforcement, and business. The goal is to identify some common approaches and principles; it is hoped that a fact-based understanding of the past will help in generating realistic expectations for the future.

Some people ask – rightly – if there is such a thing as a “lesson of history.” Of course, each situation in the past, as well as each new set of circumstances in the future, is unique and different. Details of what we learn from one situation may not be applicable to other situations, especially in a large number of cases. That said, there are similar problems over time, such as fast-moving situations, tightly held decisions made by adversaries, implications of new technologies, and skilled deception.

Technological developments in the first third of the twentieth century brought an entirely new military capability: the long-range strategic bomber that could deliver a large payload over hundreds of miles. Advocates of strategic bombing claimed that this capability had the potential to defeat an enemy without the traditional need to destroy forces in the field. Now commanders could hope to achieve a strategic goal of striking at the enemy's capacity to wage war, such as industrial production, transportation links, and the population. After the fall of France in the spring of 1940, strategic bombing was the only way Britain could hope to inflict damage on Germany, until a cross-Channel invasion could be organized.

During World War II, the main problem for Allied bombers was one of efficiency, or how to have the most impact on the German war effort with the fewest bombs. The solution was to target a limited number of facilities based on their importance to the German war economy and their vulnerability. Analysis was done in two steps: Suitable targets were identified, then results of the bombing were evaluated to determine if further strikes were necessary. Intelligence analysis could play a crucial role in this process, but British and American targeters came up with quite different ways to go about it.

The Royal Air Force's (RAF) doctrine for strategic bombing concentrated on massive, nighttime attacks on large targets, mainly cities.

According to the biblical account in the book of numbers, after leaving Egypt and then spending many years of tribulation in the Sinai desert, the Israelites approached the land of Canaan (modern Israel and parts of the surrounding countries), which had been promised to them by God. Before undertaking the dangerous and important task of launching an invasion, there were certain things that Moses, the commander, wanted to know. Following divine instructions, he sent the leaders of the twelve Israelite tribes on a mission to clandestinely collect information about what lay ahead. According to the King James version of the Bible, Moses' guidance to the spies was:

The twelve spies spent forty days in Canaan looking for indicators of the strengths and vulnerabilities of those who occupied the land they wanted.

The threat from soviet strategic weapons and the need to accurately assess Moscow's intentions and capabilities were never more important than in the 1962 Cuban Missile Crisis, the most dramatic episode in the decades-long confrontation between the United States and the USSR.

Soviet premier Nikita Khrushchev (1894–1971) was famous for his public bluster, but U.S. analysts did not appreciate the extent to which he was privately concerned that his country was at a disadvantage in the rivalry between the superpowers. For example, Khrushchev knew that, after a slow start, the United States had been able, by the early 1960s, to surpass the USSR in the production of strategic nuclear weapons and delivery systems. In addition to sites in the United States, these enormously destructive weapons were deployed on bases in Western Europe and Turkey, and thus, from Khrushchev's perspective, were able to surround the USSR. In addition, American officials had tried in 1961 to overthrow communism's only outpost in the Western Hemisphere, the regime of Fidel Castro (1926–) in Cuba.

To address these issues, Khrushchev launched a massive and daring undertaking in the spring of 1962. First, the USSR would publicly provide large amounts of conventional weapons to Cuba, including fighter aircraft, tanks, patrol boats, surface-to-air missiles (SAMs), and radar. Second, the Soviets would secretly send several dozen medium- and intermediate-range ballistic missiles (MRBMs and IRBMs) to Cuba, along with a large contingent of Soviet troops to protect them.

Air power had not been a decisive factor in world war i, but technological improvements in the 1920s and 1930s had made it a matter of great concern. Theorists, such as the Italian Giulio Douhet (1869–1930), believed that massive bombing attacks from the air would be the decisive weapon of the next war. In Britain, the leader of the Conservative Party, Stanley Baldwin (1867–1947), claimed that “the bomber will always get through.” Even in popular culture, movies such as The Shape of Things to Come envisioned a world in which airpower would be of decisive importance. Military planners trying to design and implement defenses, as well as civilian officials with budgetary responsibilities, grappled with how to assess a technology that had never been used extensively and was constantly advancing. The problem was particularly acute when trying to estimate the threat from Germany.

One of the provisions of the Treaty of Versailles had been that Germany was not to have an air force. Given the increasing importance of air power, however, the postwar German democratic governments fostered a limited secret aerial rearmament in defiance of the treaty. In 1933, Adolf Hitler (1889–1945), the head of the National Socialist (Nazi) Party who had made criticism of the treaty a key element of his foreign policy, became Chancellor of Germany and soon established a dictatorship. In addition to other rearmament efforts, he continued and accelerated clandestine efforts to build war planes.

In stark contrast to that of the previous 3,000 years, intelligence analysis in the nineteenth and twentieth centuries was transformed by three factors: bureaucracy, technology, and psychology.

From the nineteenth century onward, support to decision making in government was increasingly done by large, permanent organizations. These large organizations were characterized by hierarchical structure and division of labor, and they had both strengths and weaknesses. As demonstrated in factories, they could handle massive volumes of work, especially of a routine nature, but the flow of information up and down through the various layers of hierarchy could also lead to distortion and delay. Soon the bureaucratic style of organization was being used in armies, governments, political parties, and other functions. It was only a matter of time before it spread to intelligence analysis.

Another significant development in the nineteenth century was the increasing application of science to business, weaponry, law enforcement, and other fields. There were various versions of the scientific method, but basically the main steps were to formulate a question, propose one or more hypotheses that could answer the question, gather relevant facts, use those facts to test the hypotheses, and then accept, adjust, or abandon the hypotheses based on the results of the tests. This procedure produced a number of benefits, including new medicines and commercial products. Such systematic inquiry could also be of assistance in intelligence analysis.

On september 25, 1992, china tested a new type of nuclear bomb. For the next several years, analysts and scientists in the United States studied data on the test; in time, scientists at the Los Alamos nuclear labs, where the first U.S. nuclear weapons had been developed, became worried that the test showed that the Chinese were developing dramatically smaller warheads. If true, this represented a dramatic potential increase in China's nuclear capabilities. Miniaturized warheads could go on mobile missiles that would be harder to detect, and eventually they could be used as multiple warheads on strategic intercontinental missiles. The scientists were concerned that the Chinese could only have made such progress if they had based their warhead designs, in part, on nuclear weapons technology stolen from the United States. These fears appeared to be confirmed by a 1995 Central Intelligence Agency (CIA) report, based on 1988 documents, stating that the new Chinese bomb was similar to American technology.

Counterespionage investigations are among the most daunting challenges for intelligence analysts. It can be difficult to determine for certain, especially over the short term, whether espionage has even taken place. Spies are often experienced professionals who know how to cover their tracks. Even if there are well-founded suspicions, coming up with evidence that can hold up in court can be demanding. In any case, the investigation is almost certain to require considerable amounts of time, money, and expertise, with an uncertain outcome.

According to history of the peloponnesian war, by thucydides (c. 460–c. 395), who was an eyewitness, there was a stalemate in the war in the late fifth century BCE. After a decade and a half of fighting between Athens and its allies on one side, and Sparta along with its allies on the other, neither side had prevailed and there was a shaky peace in place on the Greek mainland. Then a plea for help arrived from a small city–state in western Sicily, Egesta (also rendered as Segesta), which was an Athenian ally. Egesta was doing poorly in a local conflict with its neighbor Selinus, which was backed by Syracuse, the dominant city–state in the eastern part of the island and a potential ally of Sparta. To make the request for assistance attractive, the Egestaeans offered to pay for Athenian support.

Athens had a democratic government, and when the appeal from Egesta had arrived in Athens there was a vigorous public debate in the assembly about how to respond. Some saw this as an opportunity to expand Athenian influence on Sicily and undermine Sparta. Others were wary about making a serious commitment so far away – Sicily was 600 miles (960 kilometers) from Athens – while the rivalry with Sparta had not been resolved. As the debate continued, it became clear that there was an important question that could be addressed by intelligence analysis: The Egestaeans had offered to make a large financial contribution, but did they really have the means to provide significant assistance?

On october 2, 2001, a photo editor for american media inc. checked into a hospital in Boca Raton, Florida (which is not far from Fort Lauderdale and Delray Beach), complaining of flulike symptoms. After a series of tests, doctors diagnosed that he had inhaled spores of anthrax, a disease not seen in its natural form in the United States since 1976, and one of the most destructive biological warfare agents. The photo editor died on October 5, and, in the meantime, a second employee of American Media was hospitalized with symptoms of inhalation anthrax. An investigation revealed that the anthrax spores had reached the American Media offices through the mail, but the original letter and envelope had since been destroyed, and it was impossible to determine any further details about who had mailed it. Postal officials decided against closing down the postal system because they were concerned that such a move might generate panic.

Because of the unusual nature of the disease and the timing – just three weeks after the 9/11 attacks – terrorism was the initial hypothesis. One of the employees of a tabloid published by American Media recalled that the paper had recently published an article that had mocked Osama bin Laden. Another editor at American Media had a spouse who was a real estate agent who had rented apartments to several of the future hijackers.