15 Short lesson: Regular Expressions
15.1 Tidyverse string expressions
When we load the Tidyverse, as above, one of the libraries included is stringr,
which provides a set of functions for working with strings.
These seem decieptively simple, but they have some powerful features that we’ll want to look at. Let’s start by reviewing the basics.
For this example, we’ll look at a short Wikipedia article. We can download it and load its contents using the following code:
## [1] "\nOn 26 July 1184, Henry VI, King of Germany (later Holy Roman Emperor), held a Hoftag (informal assembly) at the cathedral provostry in Erfurt. The combined weight of the assembled nobles caused the wooden second storey floor of the building to collapse. Most of the attendants fell through into the latrine cesspit below the ground floor, where about 60 of them drowned in liquid excrement. This event is called the Erfurter Latrinensturz (lit. 'Erfurt latrine fall') in several German sources.[1][2][3]A feud between Landgrave Louis III of Thuringia and Archbishop Conrad of Mainz, which had existed since the defeat of Henry the Lion, intensified to the point that King Henry VI was forced to intervene while he was traveling through the region during a military campaign against Poland. Henry decided to call a diet in Erfurt, where he was staying, to mediate the situation between the two, and invited a number of other figures to the negotiations.[4]All of the nobles across the Holy Roman Empire were invited to the meeting, and many arrived on 25 July to attend.[5] Just as the assembly began, the wooden floor of the deanery, in which the nobles were sitting, broke under the stress, and people fell down through the first floor into the latrine in the cellar. About 60 people are said to have died,[6] including Count Gozmar III of Ziegenhain, Count Friedrich I of Abenberg [de], Burgrave Friedrich I of Kirchberg [de], Count Heinrich I of Schwarzburg [de], Count Burgrave Burchard von Wartburg, Count Hansteiner of Liechtenstein and Beringer von Wellingen.[7][8]King Henry was said to have survived only because he sat in an alcove with a stone floor[5] and was later saved using ladders. He departed as soon as possible. Landgrave Louis III of Thuringia survived as well.[5]Of those who died, many drowned in human excrement or suffocated from the fumes emitted by the decomposing waste, while others were crushed by falling debris.\n"Some of the most important stringr functions are as follows:
str_detect() simply returns a TRUE or FALSE depending on whether the pattern
is found in the string.
## [1] TRUEstr_replace() and str_replace_all() replace the pattern with a new string.
## [1] "\nOn 26 July 1184, Henry VI, King of Germany (later Holy Roman Emperor), held a Hoftag (informal assembly) at the cathedral provostry in Zurich. The combined weight of the assembled nobles caused the wooden second storey floor of the building to collapse. Most of the attendants fell through into the latrine cesspit below the ground floor, where about 60 of them drowned in liquid excrement. This event is called the Zuricher Latrinensturz (lit. 'Zurich latrine fall') in several German sources.[1][2][3]A feud between Landgrave Louis III of Thuringia and Archbishop Conrad of Mainz, which had existed since the defeat of Henry the Lion, intensified to the point that King Henry VI was forced to intervene while he was traveling through the region during a military campaign against Poland. Henry decided to call a diet in Zurich, where he was staying, to mediate the situation between the two, and invited a number of other figures to the negotiations.[4]All of the nobles across the Holy Roman Empire were invited to the meeting, and many arrived on 25 July to attend.[5] Just as the assembly began, the wooden floor of the deanery, in which the nobles were sitting, broke under the stress, and people fell down through the first floor into the latrine in the cellar. About 60 people are said to have died,[6] including Count Gozmar III of Ziegenhain, Count Friedrich I of Abenberg [de], Burgrave Friedrich I of Kirchberg [de], Count Heinrich I of Schwarzburg [de], Count Burgrave Burchard von Wartburg, Count Hansteiner of Liechtenstein and Beringer von Wellingen.[7][8]King Henry was said to have survived only because he sat in an alcove with a stone floor[5] and was later saved using ladders. He departed as soon as possible. Landgrave Louis III of Thuringia survived as well.[5]Of those who died, many drowned in human excrement or suffocated from the fumes emitted by the decomposing waste, while others were crushed by falling debris.\n"str_count() counts the number of times the pattern appears in the string.
## [1] 3str_extract() and str_extract_all()
pull every time the pattern is mentioned in the string.
## [[1]]
## [1] "King" "King" "King"But this one is a little puzzling; why would we ever need to do this?
15.2 Regular Expressions
The answer is that there are some special codes that we can put into the pattern to return some more complicated results. These are called regular expressions, or regex for short.
These are incredibly powerful, but sometimes a little hard to wrap your mind around. A one simple example is \d, which matches any digit. So we can use this to find every 0-9 in the string.
## [[1]]
## [1] "2" "6" "1" "1" "8" "4" "6" "0" "1" "2" "3" "4" "2" "5" "5" "6" "0" "6" "7" "8" "5" "5"## [[1]]
## [1] "2" "6" "1" "1" "8" "4" "6" "0" "1" "2" "3" "4" "2" "5" "5" "6" "0" "6" "7" "8" "5" "5"15.3 Counting Counts
In the middle of this article, we have a list of royalty.
...
including Count Gozmar III of Ziegenhain, Count Friedrich I of Abenberg [de],
Burgrave Friedrich I of Kirchberg [de], Count Heinrich I of Schwarzburg [de],
Count Burgrave Burchard of Wartburg [de], Count Hansteiner of Liechtenstein,
...As a way to practice regex, let’s build up a way to extract all of these names.
First, we can see that all of this royalty follows a common pattern; an expression that is regular, if you will.
Basically, each one goes:
TITLE Name Sometimes a number of Place
15.3.1 Choices
We can start from the beginning, and get all the titles. With a regex, we can use the () parentheses and | pipe to select any of the possible titles.
## [[1]]
## [1] "Landgrave" "Archbishop" "Count" "Count" "Burgrave" "Count" "Count" "Burgrave" "Count" "Landgrave"15.3.2 Wildcards
This is a good start! But we need the full title. like \\d,
there are some other wildcards that can help us get the next parts of the name.
\\wMatches anything that isn’t a space.
## [[1]]
## [1] "H" "e" "n" "r" "y" "V" "I" "I" "I"\\sMatches any kind of white space.
## [1] " "[ABC]Matches any of the characters A, B, or C.
## [[1]]
## [1] "V" "I" "I" "I"So, we can get the next letters by saying TITLE+SPACE+LETTER
## [[1]]
## [1] "Landgrave L" "Archbishop C" "Count G" "Count F" "Burgrave F" "Count H" "Count B" "Count H" "Landgrave L"Better than before, but we can do better.
15.3.3 Repetition
If you want more than 1 of something in Regex,
?Matches 0 or 1 of something.+Matches 1 or more of something.*Matches 0 or more.{3}Matches exactly 3 of something{3,}Matches 3 or more{5,6}Matches between 5 and 6 of something.
These are super confusing! The best thing to do is just get an example and play around with them.
We can then get the full name of our royalty by doing something TITLE+SPACE+[More than 1 letter]
## [[1]]
## [1] "Landgrave Louis" "Archbishop Conrad" "Count Gozmar" "Count Friedrich" "Burgrave Friedrich" "Count Heinrich" "Count Burgrave" "Count Hansteiner" "Landgrave Louis"Now, Some of these royals have ordinals after their name, but not all of them! So we will need
to use the * to capture zero or more roman numerals, like so:
TITLE+SPACE+[More than 1 letter]+[zero or more spaces]+[zero or more roman numerals]
## [[1]]
## [1] "Landgrave Louis III" "Archbishop Conrad " "Count Gozmar III" "Count Friedrich I" "Burgrave Friedrich I" "Count Heinrich I" "Count Burgrave " "Count Hansteiner " "Landgrave Louis III"Finally, We need the last word, which occurs on all of them. We just add two more words to the regex:
TITLE+SPACE+[More than 1 letter]+[zero or more spaces]+[zero or more roman numerals]+of+[More than 1 letter]
str_extract_all(erfurt, "(Count|Archbishop|Burgrave|Burgmeister|Landgrave)\\s\\w+\\s*[IVXLCDM]*\\sof\\s\\w+")## [[1]]
## [1] "Landgrave Louis III of Thuringia" "Archbishop Conrad of Mainz" "Count Gozmar III of Ziegenhain" "Count Friedrich I of Abenberg" "Burgrave Friedrich I of Kirchberg" "Count Heinrich I of Schwarzburg" "Count Hansteiner of Liechtenstein"
## [8] "Landgrave Louis III of Thuringia"And there we have it! All the victims of the Erfurt latrine disaster.
15.4 A Warning
Many people consider Regexes to be “read-only code”; once you’ve written it, you’ll never be able to go back and understand it. You should really only use them when you need to do something very specific.
15.5 Resources
Regexes are hard, and sometimes you might feel like you’re experiencing your own little Erfurt latrine disaster.
Here are a few things that can help you get out of the pit:
A really complete introduction to Regex in R can be found on Stringr documentation here:
https://stringr.tidyverse.org/articles/regular-expressions.html
An interactive console for regexes can be found here:
Note that R is not officially supported here, but you can use Python,
then change all the single backslashes to doubles, and it will mostly work in R (e.g. \d to \\d)
15.6 Practice
Here’s the text of a Wikipedia article.
us_history <- read_html("https://en.wikipedia.org/wiki/History_of_the_United_States")
us_history <- us_history |>
html_elements('p') |>
html_text() |>
paste(collapse = " ")
us_history## [1] "This is an accepted version of this page \n \n\n\n The history of the lands that became the United States began with the arrival of the first people in the Americas around 15,000 BC. After European colonization of North America began in the late 15th century, wars and epidemics decimated Indigenous societies. By the 1760s, the thirteen British colonies were established. The Southern Colonies built an agricultural system on slave labor, enslaving millions from Africa. After defeating France, the British Parliament imposed a series of taxes; resistance to these taxes, especially the Boston Tea Party in 1773, led to Parliament issuing the Intolerable Acts designed to end self-government.\n In 1776, the United States declared its independence. Led by General George Washington, it won the Revolutionary War in 1783. The Constitution was adopted in 1789, and a Bill of Rights was added in 1791 to guarantee inalienable rights. Washington, the first president, and his adviser Alexander Hamilton created a strong central government. The Louisiana Purchase in 1803 doubled the size of the country. Encouraged by available, inexpensive land and the notion of manifest destiny, the country expanded to the Pacific Coast. The resulting expansion of slavery was increasingly controversial. After the election of Abraham Lincoln as president in 1860, the southern states seceded to form the pro-slavery Confederate States of America, and started the Civil War. The Confederates' defeat in 1865 led to the abolition of slavery. In the subsequent Reconstruction era, the national government gained explicit duty to protect individual rights. White southern Democrats regained their political power in the South in 1877, often using paramilitary suppression of voting and Jim Crow laws to maintain white supremacy.\n The United States became the world's leading industrial power in the 20th century, largely due to entrepreneurship, industrialization, and the arrival of millions of immigrant workers. A national railroad network was completed, and large-scale mines and factories were established. Dissatisfaction with corruption, inefficiency, and traditional politics stimulated the Progressive movement, leading to reforms including the federal income tax, direct election of Senators, citizenship for many Indigenous people, alcohol prohibition, and women's suffrage. Initially neutral during World War I, the United States declared war on Germany in 1917, joining the successful Allies. After the prosperous Roaring Twenties, the Wall Street crash of 1929 marked the onset of the decade-long worldwide Great Depression. President Franklin D. Roosevelt's New Deal programs, including unemployment relief and social security, defined modern American liberalism.[1] Following the Japanese attack on Pearl Harbor, the United States entered World War II, helping defeat Nazi Germany and Fascist Italy in the European theater. In the Pacific War, America defeated Imperial Japan after using nuclear weapons on Hiroshima and Nagasaki.\n The United States and the Soviet Union emerged as rival superpowers during the Cold War; the two countries confronted each other indirectly in the arms race, the Space Race, propaganda campaigns, and proxy wars. In the 1960s, in large part due to the civil rights movement, social reforms enforced the constitutional rights of voting and freedom of movement to African Americans. In the 1980s, Ronald Reagan's presidency realigned American politics towards reductions in taxes and regulations. The Cold War ended when the Soviet Union was dissolved in 1991, leaving the United States as the world's sole superpower. Foreign policy after the Cold War has often focused on conflicts in the Middle East, especially after the September 11 attacks. In the 21st century, the country was negatively affected by the Great Recession and the COVID-19 pandemic.\n It is not definitively known how or when Native Americans first settled the Americas. The prevailing theory proposes that people from Eurasia followed game across Beringia, a land bridge that connected Siberia to present-day Alaska during the Ice Age, and then spread southward, perhaps as early as 30,000 years ago.[2] These early inhabitants, called Paleo-Indians, soon diversified into hundreds of culturally distinct groups.\n By 10,000 BCE, humans were well-established throughout North America. Originally, Paleo-Indians hunted Ice Age megafauna like mammoths, but as they began to go extinct, people turned instead to bison as a food source, and later foraging for berries and seeds. Paleo-Indians in central Mexico were the first in the Americas to farm, around 8,000 BCE. Eventually, the knowledge began to spread northward. By 3,000 BCE, corn was being grown in the valleys of Arizona and New Mexico, followed by primitive irrigation systems and, by 300 BCE, early villages of the Hohokam.[3][4] One of the earlier cultures in the present-day United States was the Clovis culture (9,100 to 8,850 BCE), who are primarily identified by the use of fluted spear points called the Clovis point. The Folsom culture was similar, but is marked by the use of the Folsom point.\n A later migration around 8,000 BCE included Na-Dene-speaking peoples, who reached the Pacific Northwest by 5,000 BCE.[5] From there, they migrated along the Pacific Coast and into the interior.[6] Another group, the Oshara tradition people, who lived from 5,500 BCE to 600 CE, were part of the Archaic Southwest.\n The Adena began constructing large earthwork mounds around 600 BCE. They are the earliest known people to have been Mound Builders, although there are mounds in the United States that predate this culture. The Adenans were absorbed into the Hopewell tradition, a powerful people who traded tools and goods across a wide territory. They continued the Adena tradition of mound-building and pioneered a trading system called the Hopewell Exchange System, which at its greatest extent ran from the present-day Southeast up to the Canadian side of Lake Ontario.[7] By 500 CE, the Hopewellians had been absorbed into the larger Mississippian culture.\n The Mississippians were a broad group of tribes. Their most important city was Cahokia, near modern-day St. Louis, Missouri. At its peak in the 12th century, the city had an estimated population of 20,000, larger than the population of London at the time. The entire city was centered around a mound that stood 100 feet (30 m) tall. Cahokia, like many other cities and villages of the time, depended on hunting, foraging, trading, and agriculture, and developed a class system with slaves and human sacrifice that was influenced by societies to the south, like the Mayans.[3] In the Southwest, the Anasazi began constructing stone and adobe pueblos around 900 BCE.[8] These apartment-like structures were often built into cliff faces, as seen in the Cliff Palace at Mesa Verde. Some grew to be the size of cities, with Pueblo Bonito along the Chaco River in New Mexico once consisting of 800 rooms.[3] The Indigenous peoples of the Pacific Northwest were likely the most affluent Native Americans. Many distinct cultural groups and political entities developed there, but they all shared certain beliefs, traditions, and practices, such as the centrality of salmon as a resource and spiritual symbol. Permanent villages began to develop in this region as early as 1,000 BCE, and these communities celebrated by the gift-giving feast of the potlatch.\n In present-day upstate New York, the Iroquois formed a confederacy of tribal peoples in the mid-15th century, consisting of the Oneida, Mohawk, Onondaga, Cayuga, and Seneca.[9][10][11] Each tribe had seats in a group of 50 sachem chiefs. It has been suggested that their culture contributed to political thinking during the development of the United States government.[citation needed] The Iroquois were powerful, waging war with many neighboring tribes, and later, Europeans. As their territory expanded, smaller tribes were forced further west, including the Osage, Kaw, Ponca, and Omaha peoples.[11][12] The exact date for the settling of Hawaii is disputed but the first settlement most likely took place between 940 and 1130 CE.[13] Around 1200 CE, Tahitian explorers found and began settling the area. This marked the rise of the Hawaiian civilization, which would be largely separated from the rest of the world until the arrival of the British 600 years later.[14][15][16] Europeans under the British explorer James Cook arrived in the Hawaiian Islands in 1778, and within five years of contact, European military technology would help Kamehameha I conquer most of the island group, and eventually unify the islands for the first time, establishing the Hawaiian Kingdom.[17] The island of Puerto Rico has been settled for at least 4,000 years. Starting with the Ortoiroid culture, successive generations of native migrations arrived replacing or absorbing local populations. By the year 1000 Arawak people had arrived from South America via the Lesser Antilles; these settlers would become the Taíno encountered by the Spanish in 1493. Upon European contact a native population between 30,000 and 60,000 was likely, led by a single chief called a Cacique.[18] Colonization resulted in the decimation of the local inhabitants due to the harsh Encomienda system and epidemics caused by Old World diseases. Puerto Rico would remain a part of Spain until American annexation in 1898.[18] The earliest recorded European mention of America is in a treatise by the medieval chronicler Adam of Bremen, circa 1075, where it is referred to as Vinland.[a] It is also extensively referred to in the Norse Vinland sagas. The strongest archaeological evidence of the existence of Norse settlements in America is located in Canada; there is significant scholarly debate as to whether Norse explorers also made landfall in New England.[20] Europeans brought horses, cattle, and hogs to the Americas and took back maize, turkeys, tomatoes, potatoes, tobacco, beans, and squash to Europe. Many explorers and early settlers died after being exposed to new diseases in the Americas. However, the effects of new Eurasian diseases carried by the colonists, especially smallpox and measles, were much worse for the Native Americans, as they had no immunity to them. They suffered epidemics and died in very large numbers, usually before large-scale European settlement began. Their societies were disrupted by the scale of deaths.[21][22] Spanish explorers were the first Europeans, after the Norse, to reach the present-day United States, after the voyages of Christopher Columbus (beginning in 1492) established possessions in the Caribbean, including the modern-day U.S. territories of Puerto Rico, and parts of the U.S. Virgin Islands. Juan Ponce de León landed in Florida in 1513.[23] Spanish expeditions quickly reached the Appalachian Mountains, the Mississippi River, the Grand Canyon,[24] and the Great Plains.[25] In 1539, Hernando de Soto extensively explored the Southeast,[25] and a year later Francisco Coronado explored from Arizona to central Kansas in search of gold.[25] Escaped horses from Coronado's party spread over the Great Plains, and the Plains Indians mastered horsemanship within a few generations.[3] Small Spanish settlements eventually grew to become important cities, such as San Antonio, Albuquerque, Tucson, Los Angeles, and San Francisco.[26] The Dutch East India Company sent explorer Henry Hudson to search for a Northwest Passage to Asia in 1609. New Netherland was established in 1621 by the company to capitalize on the North American fur trade. Growth was slow at first due to mismanagement by the Dutch and Native American conflicts. After the Dutch purchased the island of Manhattan from the Native Americans, the land was named New Amsterdam and became the capital of New Netherland. The town rapidly expanded and in the mid-1600s it became an important trading center. Despite being Calvinists and building the Reformed Church in America, the Dutch were tolerant of other religions and cultures and traded with the Iroquois to the north.[27] The colony served as a barrier to British expansion from New England, and as a result a series of wars were fought. The colony was taken over by Britain as New York in 1664 and its capital was renamed New York City.\n In the early years of the Swedish Empire, Swedish, Dutch, and German stockholders formed the New Sweden Company to trade furs and tobacco in North America. The company's first expedition was led by Peter Minuit, who had been governor of New Netherland from 1626 to 1631, and landed in Delaware Bay in March 1638. The settlers founded Fort Christina at the site of modern-day Wilmington, Delaware, and made treaties with Indigenous peoples for land ownership on both sides of the Delaware River.[28][29] Over the following seventeen years, 12 more expeditions brought settlers from the Swedish Empire to New Sweden. The colony established 19 permanent settlements along with many farms, extending into modern-day Maryland, Pennsylvania, and New Jersey. It was incorporated into New Netherland in 1655 after a Dutch invasion from the neighboring New Netherland colony during the Second Northern War.[28][29] Giovanni da Verrazzano landed in North Carolina in 1524, and was the first European to sail into New York Harbor and Narragansett Bay. In the 1540s, French Huguenots settled at Fort Caroline near present-day Jacksonville, Florida. In 1565, Spanish forces led by Pedro Menéndez destroyed the settlement and established the first Spanish settlement in what would become the United States — St. Augustine.\n Most French lived in Quebec and Acadia (modern Canada), but far-reaching trade relationships with Native Americans spread their influence. French colonists in small villages along the Mississippi and Illinois rivers lived in farming communities that served as a grain source for Gulf Coast settlements. The French established plantations in Louisiana along with settling New Orleans, Mobile and Biloxi.\n The English, drawn in by Francis Drake's raids on Spanish treasure ships leaving the New World, settled the strip of land along the east coast in the 1600s.[3] The early British colonies were established by private groups seeking profit, and were marked by starvation, disease, and Native American attacks. Many immigrants were people seeking religious freedom or escaping political oppression, peasants displaced by the Industrial Revolution, or those simply seeking adventure and opportunity. Between the late 1610s and the Revolution, the British shipped an estimated 50,000 to 120,000 convicts to their American colonies.[30] In some areas, Native Americans taught colonists how to grow local crops. In others, they attacked the settlers. Virgin forests provided an ample supply of building material and firewood. Natural inlets and harbors lined the coast, providing easy ports for essential trade with Europe. Settlements remained close to the coast due to this as well as Native American resistance and the Appalachian Mountains in the interior.[3] The first successful English colony, Jamestown, was established by the Virginia Company in 1607 on the James River in Virginia. The colonists were preoccupied with the search for gold and were ill-equipped for life in the New World. Captain John Smith held the fledgling Jamestown together in the first year, and the colony descended into anarchy and nearly failed when he returned to England two years later. John Rolfe began experimenting with tobacco from the West Indies in 1612, and by 1614 the first shipment arrived in London. It became Virginia's chief source of revenue within a decade.\n In 1624, after years of disease and Indian attacks, including the Powhatan attack of 1622, King James I revoked the Virginia Company's charter and made Virginia a royal colony.\n New England was initially settled primarily by Puritans fleeing religious persecution. The Pilgrims sailed for Virginia on the Mayflower in 1620, but were knocked off course by a storm and landed at Plymouth, where they agreed to a social contract of rules in the Mayflower Compact. About half died in the first winter.[31] Like Jamestown, Plymouth suffered from disease and starvation, but local Wampanoag Indians taught the colonists how to farm maize.\n Plymouth was followed by the Puritans and Massachusetts Bay Colony in 1630. They maintained a charter for self-government separate from England, and elected founder John Winthrop as governor. Roger Williams opposed Winthrop's treatment of Native Americans and religious intolerance, and established the colony of Providence Plantations, later Rhode Island, on the basis of freedom of religion. Other colonists established settlements in the Connecticut River Valley, and on the coasts of present-day New Hampshire and Maine. Native American attacks continued, with the most significant occurring in the 1637 Pequot War and the 1675 King Philip's War.\n New England became a center of commerce and industry due to the poor, mountainous soil making agriculture difficult. Rivers were harnessed to power grain mills and sawmills, and the numerous harbors facilitated trade. Tight-knit villages developed around these industrial centers, and Boston became one of America's most important ports.\n In the 1660s, the Middle Colonies of New York, New Jersey, and Delaware were established in the former Dutch New Netherland, and were characterized by a large degree of ethnic and religious diversity. At the same time, the Iroquois of New York, strengthened by years of fur trading with Europeans, formed the powerful Iroquois Confederacy.\n The last colony in this region was Pennsylvania, established in 1681 by William Penn as a home for religious dissenters, including Quakers, Methodists, and the Amish.[32] The capital of the colony, Philadelphia, became a dominant commercial center in a few short years. While Quakers populated the city, German immigrants began to flood into the Pennsylvanian hills and forests, while the Scots-Irish pushed into the far western frontier.\n The overwhelmingly rural Southern Colonies contrasted sharply with the New England and Middle Colonies. After Virginia, the second British colony south of New England was Maryland, established as a Catholic haven in 1632. The economy of these two colonies was built entirely on yeoman farmers and planters. The planters established themselves in the Tidewater region of Virginia, establishing massive plantations with slave labor.\n In 1670, the Province of Carolina was established, and Charleston became the region's great trading port. While Virginia's economy was also based on tobacco, Carolina was more diversified, exporting rice, indigo, and lumber as well. In 1712, it was divided in two, creating North and South Carolina. The Georgia Colony was established by James Oglethorpe in 1732 as a border to Spanish Florida and a reform colony for former prisoners and the poor.[32] Religiosity expanded greatly after the First Great Awakening, a religious revival in the 1740s led by preachers such as Jonathan Edwards and George Whitefield. American Evangelicals affected by the Awakening added a new emphasis on divine outpourings of the Holy Spirit and conversions that implanted new believers with an intense love for God. Revivals encapsulated those hallmarks and carried the newly created evangelicalism into the early republic, setting the stage for the Second Great Awakening in the late 1790s.[33] In the early stages, evangelicals in the South, such as Methodists and Baptists, preached for religious freedom and abolition of slavery.\n Each of the American colonies had a slightly different governmental structure. Typically, a colony was ruled by a governor appointed from London who controlled the executive administration and relied upon a locally elected legislature to vote on taxes and make laws. By the 18th century, the American colonies were growing very rapidly as a result of low death rates along with ample supplies of land and food. The colonies were richer than most parts of Britain, and attracted a steady flow of immigrants, especially teenagers who arrived as indentured servants.[34] Over half of all European immigrants to Colonial America arrived as indentured servants.[35] Typically, people would sign a contract agreeing to a set term of labor, usually four to seven years, and in return would receive transport to America and a piece of land at the end of their servitude. In some cases, ships' captains received rewards for the delivery of poor migrants, and so extravagant promises and kidnapping were common.[3] The first African slaves arrived in 1619.[36] Initially regarded as indentured servants who could buy their freedom, the institution of slavery began to harden and the involuntary servitude became lifelong[36] as the demand for labor on tobacco and rice plantations grew in the 1660s.[citation needed] Slavery became identified with brown skin color, and the children of slave women were born slaves (partus sequitur ventrem).[36] By the 1770s African slaves comprised a fifth of the American population.\n The question of independence from Britain did not arise as long as the colonies needed British military support against the French and Spanish powers. Those threats were gone by 1765. However, London continued to regard the American colonies as existing for the benefit of the mother country in a policy known as mercantilism.[34] Colonial America was defined by a severe labor shortage that used forms of unfree labor, such as slavery and indentured servitude. The British colonies were also marked by a policy of avoiding strict enforcement of parliamentary laws, known as salutary neglect. This permitted the development of an American spirit distinct from that of its European founders.[37] The French and Indian War (1754–1763), part of the larger Seven Years' War, was a watershed event in the political development of the colonies. The influence of the French and Native Americans, the main rivals of the British Crown in the colonies and Canada, was significantly reduced and the territory of the Thirteen Colonies expanded into New France in Canada and Louisiana.[citation needed] The war effort also resulted in greater political integration of the colonies, as reflected in the Albany Congress and symbolized by Benjamin Franklin's call for the colonies to \"Join, or Die.\"[38] King George III issued the Royal Proclamation of 1763, to organize the new North American empire and protect the Native Americans from colonial expansion west of the Appalachian Mountains. Strains developed in the relations between the colonists and the Crown. The British Parliament passed the Stamp Act of 1765, imposing a tax on the colonies without going through the colonial legislatures. Crying \"No taxation without representation\", the colonists refused to pay.[39] The Boston Tea Party in 1773 was a direct action to protest the new tax on tea. Parliament responded the next year with the Intolerable Acts, stripping Massachusetts of its historic right of self-government and putting it under military rule, which sparked outrage and resistance in all thirteen colonies. Patriot leaders from every colony convened the First Continental Congress to coordinate their resistance. The Congress called for a boycott of British trade, published a list of rights and grievances, and petitioned the king to rectify those grievances.[40] This appeal had no effect.\n The Second Continental Congress voted to declare independence on July 2, 1776. The Declaration of Independence presented arguments in favor of the rights of citizens, stating that all men are created equal, supporting the rights of Life, Liberty and the pursuit of Happiness, and demanding the consent of the governed.[41] The Founding Fathers were guided by the ideology of republicanism, rejecting the monarchism of Great Britain.[42] The Declaration of Independence was signed by members of the Congress on July 4.[41] This date has since been commemorated as Independence Day.[43] The American Revolutionary War began with the Battles of Lexington and Concord on April 19, 1775.[44]George Washington was appointed general of the Continental Army.[45] Washington's crossing of the Delaware River began a series of victories that expelled British forces from New Jersey.[46] The British began the Saratoga campaign in 1777 to capture Albany, New York, as a choke point.[47] After American victory at Saratoga, France, the Netherlands, and Spain began providing support to the Continental Army.[48] Britain responded to defeat in the northern theater by advancing in the southern theater, beginning with the Capture of Savannah in 1778.[49] American forces reclaimed the south in 1781, and the British Army was defeated in the Siege of Yorktown on October 19, 1781.[50] King George III formally ordered the end of hostilities on December 5, 1782, recognizing American independence.[51] The Treaty of Paris was signed on September 3, 1783,[52] and was ratified by the Congress of the Confederation on January 14, 1784.[53] The Articles of Confederation were ratified as the governing law of the United States, written to limit the powers of the central government in favor of states. This caused economic decline, as the government was unable to pass economic legislation and pay its debts.[54] Nationalists worried that the confederate nature of the union was too fragile to withstand an armed conflict with any adversarial states, or even internal revolts such as Shays' Rebellion of 1786 in Massachusetts.[55] In the 1780s the western regions were ceded by the states to Congress and became territories. With the migration of settlers to the Northwest, soon they became states.[55] The American Indian Wars continued in the 1780s as settlers moved west.[56] The Northwestern Confederacy and American settlers began fighting the Northwest Indian War in the late 1780s; the Northwestern Confederacy received British support, but the settlers received little assistance from the American government.[57][58] Nationalists – most of them war veterans – organized in every state and convinced Congress to call the Philadelphia Convention in 1787. The delegates from every state wrote a new Constitution that created a federal government with a strong president and powers of taxation. The new government reflected the prevailing republican ideals of guarantees of individual liberty and of constraining the power of government through separation of powers.[55] The constitution was ratified by a sufficient number of states in 1788 to begin forming a federal government.[59] The United States Electoral College chose George Washington as the first President in 1789.[60] The national capital moved from New York to Philadelphia in 1790 and finally to Washington, D.C., in 1800.\n The major accomplishment of the Washington Administration was creating a strong national government that was recognized by all Americans.[61] His government, following the vigorous leadership of Treasury Secretary Alexander Hamilton, assumed the debts of the states, created the Bank of the United States, and set up a uniform system of tariffs and other taxes to pay off the debt and provide a financial infrastructure. To support his programs Hamilton created the Federalist Party. To assuage the Anti-Federalists who feared a too-powerful central government, the Congress adopted the United States Bill of Rights in 1791, which guaranteed individual liberties such as freedom of speech and religious practice.[62] Thomas Jefferson and James Madison formed an opposition Republican Party (usually called the Democratic-Republican Party). Hamilton and Washington presented the country in 1794 with the Jay Treaty that reestablished good relations with Britain. The Jeffersonians vehemently protested, and the voters aligned behind one party or the other, thus setting up the First Party System.[63] The treaty passed, but politics became intensely heated.[64] Serious challenges to the new federal government included the Northwest Indian War, the ongoing Cherokee–American wars, and the 1794 Whiskey Rebellion, in which western settlers protested against a federal tax on liquor.[65]\nWashington refused to serve more than two terms – setting a precedent.[66]John Adams, a Federalist, defeated Jefferson in the 1796 election. War loomed with France and the Federalists used the opportunity to try to silence the Republicans with the Alien and Sedition Acts, build up a large army with Hamilton at the head, and prepare for a French invasion. However, the Federalists became divided after Adams sent a successful peace mission to France that ended the Quasi-War of 1798.[63][67] During the first two decades after the Revolutionary War, there were dramatic changes in the status of slavery among the states and an increase in the number of freed blacks. Inspired by revolutionary ideals of equality and influenced by their lesser economic reliance on slavery, northern states abolished slavery.\n States of the Upper South made manumission easier, resulting in an increase in the proportion of free blacks in the Upper South (as a percentage of the total non-white population) from less than one percent in 1792 to more than 10 percent by 1810. By that date, a total of 13.5 percent of all blacks in the United States were free.[68] In 1807, with four million slaves already in the United States, Congress severed U.S. involvement with the Atlantic slave trade.[69] The Second Great Awakening was a Protestant revival movement that affected virtually all of society during the early 19th century and led to rapid church growth. The movement began around 1790, gained momentum by 1800, and, after 1820 membership rose rapidly among Baptist and Methodist congregations, whose preachers led the movement. It was past its peak by the 1840s.[70] It enrolled millions of new members in existing evangelical denominations and led to the formation of new denominations. The Second Great Awakening stimulated the establishment of many reform movements, including abolitionism and temperance.[71] Jefferson defeated Adams massively for the presidency in the 1800 election. Jefferson's major achievement as president was the Louisiana Purchase in 1803, which provided U.S. settlers with vast potential for expansion west of the Mississippi River.[72]\nJefferson supported expeditions to explore and map the new domain, most notably the Lewis and Clark Expedition.[73] Jefferson believed deeply in republicanism and argued it should be based on the independent yeoman farmer and planter; he distrusted cities, factories and banks. He also distrusted the federal government and judges, and tried to weaken the judiciary. Although the Constitution specified a Supreme Court, its functions were vague until John Marshall, the Chief Justice of the United States (1801–1835), defined them, especially the power to overturn acts of Congress or states that violated the Constitution, first enunciated in 1803 in Marbury v. Madison.[74] Americans were increasingly angered by the British violation of American ships' neutral rights to harm France, the coercion of 10,000 American sailors needed by the Royal Navy to fight Napoleon, and British support for hostile Indians attacking American settlers in the American Midwest with the goal of creating a pro-British Indian barrier state to block American expansion westward. They may also have wished to annex all or part of British North America, although this is still heavily debated.[75][76][77][page needed][78][79] Despite strong opposition from the Northeast, especially from Federalists who did not want to disrupt trade with Britain, Congress declared war on the United Kingdom on June 18, 1812.[80] Both sides tried to invade the other and were repulsed. The American militia proved ineffective because the soldiers were reluctant to leave home, and efforts to invade Canada repeatedly failed. The British blockade ruined American commerce, bankrupted the Treasury, and further angered New Englanders, who smuggled supplies to Britain. The Americans under General William Henry Harrison finally gained naval control of Lake Erie and defeated the Indians under Tecumseh in Canada,[82] while Andrew Jackson ended the Indian threat in the Southeast. The Indian threat to expansion into the Midwest was permanently ended. The British invaded and occupied much of Maine.\n In 1814, the British raided and burned Washington but were repelled at Baltimore, where \"The Star-Spangled Banner\" was written to celebrate the American success. In upstate New York, a major British invasion of New York State was turned back at the Battle of Plattsburgh. In early 1815, Andrew Jackson decisively defeated a major British invasion at the Battle of New Orleans,[83][page needed] and the Americans finally claimed victory on February 18, as news came almost simultaneously of Jackson's victory of New Orleans and the peace treaty that left the prewar boundaries in place. This \"second war of independence\" helped lead to an emerging American identity that cemented national pride over state pride.[84] The War of 1812 also dispelled America's negative perception of a standing army as opposed to ill-equipped and poorly-trained militias.[85] National euphoria after the victory at New Orleans ruined the prestige of the Federalists and they no longer played a significant role as a political party.[86] President Madison and most Republicans realized they were foolish to let the First Bank of the United States close down, for its absence greatly hindered the financing of the war. With the assistance of foreign bankers, they chartered the Second Bank of the United States in 1816.[87][88] The Republicans also imposed tariffs designed to protect the infant industries that had been created when Britain was blockading the U.S. With the collapse of the Federalists as a party, the adoption of many Federalist principles by the Republicans, and the systematic policy of President James Monroe in his two terms (1817–1825) to downplay partisanship, society entered an Era of Good Feelings and closed out the First Party System.[87][88] The Monroe Doctrine, expressed in 1823, proclaimed the United States' opinion that European powers should no longer colonize or interfere in the Americas. This was a defining moment in U.S. foreign policy.[89] In 1832, President Andrew Jackson ran for a second term under the slogan \"Jackson and no bank\" and did not renew the charter of the Second Bank, dissolving the bank in 1836.[90] Jackson was convinced that central banking was used by the elite to take advantage of the average American, and instead implemented publicly owned banks in various states, popularly known as \"pet banks\".[90] The former Jeffersonian (Democratic-Republican) party split into factions over the choice of a successor to President James Monroe, and the party faction that supported many of the old Jeffersonian principles, led by Andrew Jackson and Martin Van Buren, became the Democratic Party. As Norton explains the transformation in 1828:\n Jacksonians believed the people's will had finally prevailed. Through a lavishly financed coalition of state parties, political leaders, and newspaper editors, a popular movement had elected the president. The Democrats became the nation's first well-organized national party, and tight party organization became the hallmark of nineteenth-century American politics.[91] Opposing factions led by Henry Clay helped form the Whig Party. The Democratic Party had a small but decisive advantage over the Whigs until the 1850s, when the Whigs fell apart over the issue of slavery.\n The country grew rapidly in population and area, as pioneers pushed the frontier of settlement west.[92][93] Native American tribes in some places resisted militarily, but they were overwhelmed by settlers and the army, and after 1830, were relocated to reservations in the west.[94] That year, Congress passed the Indian Removal Act, which authorized the president to negotiate treaties that exchanged Native American tribal lands in the eastern states for lands west of the Mississippi River.[95] Its goal was primarily to remove Native Americans, including the Five Civilized Tribes, from desirable lands in the American Southeast.[96] Thousands of deaths resulted from the relocations, as seen in the Cherokee Trail of Tears,[96] which resulted in approximately 2,000 to 8,000 of the 16,543 relocated Cherokee dying along the way.[97][98] Many of the Seminole Indians in Florida refused to move west, and fought the Army for years in the Seminole Wars.\n The first settlers in the west were the Spanish in New Mexico (\"Californios\"), followed by over 100,000 California Gold Rush miners ('49ers). California grew rapidly, and by 1880, San Francisco had become the economic hub of the Pacific Coast, with a diverse population of a quarter million. From the early 1830s to 1869, the Oregon Trail and its offshoots were used by over 300,000 settlers headed to California, Oregon, and other points in the far west. Wagon trains took five or six months on foot.[99] Manifest destiny was the belief that American settlers were destined to expand across the continent.[100] Manifest destiny was rejected by modernizers, especially the Whigs like Henry Clay and Abraham Lincoln who wanted to build cities and factories – not more farms.[b]Democrats strongly favored expansion, and won the key election of 1844. After a bitter debate in Congress, the Republic of Texas was annexed in 1845, leading to the Mexican–American War.[102] The U.S. Army invaded Mexico at several points, captured Mexico City, and won the war decisively. The Treaty of Guadalupe Hidalgo ended the war in 1848. Many Democrats wanted to annex all of Mexico, but that idea was rejected by White Southerners, who argued that incorporating millions of Mexican people, mainly of mixed race, would undermine the U.S. as an exclusively white republic.[101] Instead, the U.S. took Texas and the lightly settled northern parts (California and New Mexico). Simultaneously, gold was discovered in California in 1848. To clear the state for settlers, the U.S. government began a policy of extermination since termed the California genocide.[103] A peaceful compromise with Britain gave the U.S. ownership of the Oregon Country, which was renamed the Oregon Territory.[102]\nThe demand for guano (prized as an agricultural fertilizer) led the U.S. to pass the Guano Islands Act in 1856, which enabled U.S. citizens to take possession, in the name of the country, of unclaimed islands containing guano deposits. Under the act, the U.S. annexed nearly 100 islands in the Pacific Ocean and the Caribbean Sea. By 1903, 66 of these islands were recognized as U.S. territories.[104] The women's suffrage movement began with the 1848 National Convention of the Liberty Party. Presidential candidate Gerrit Smith established women's suffrage as a party goal. One month later, the Seneca Falls Convention was organized, signing the Declaration of Sentiments demanding equal rights for women, including the right to vote.[c] The women's rights campaign during first-wave feminism was led by Elizabeth Cady Stanton, Lucy Stone and Susan B. Anthony, among others. Stone and Paulina Wright Davis organized the prominent and influential National Women's Rights Convention in 1850.[106] The central issue after 1848 was the expansion of slavery, with the anti-slavery elements in the North pitted against the pro-slavery elements in the South.[107] By 1860, there were four million slaves in the South. A small number of Northerners sought the immediate abolition of slavery. Much larger numbers in the North were against the expansion of slavery, seeking to put it on the path to extinction.[107] There were violent reactions to abolitionist advocates in the North, notably the burning of an anti-slavery society in Pennsylvania Hall.[108] There was resistance to slavery by both peaceful and violent means. Slave rebellions, by Gabriel Prosser (1800), Denmark Vesey (1822), Nat Turner (1831), and John Brown (1859) caused fear in the white South, which imposed stricter oversight of slaves and reduced the rights of free Black people.[citation needed]Southern white Democrats insisted that slavery was of economic, social, and cultural benefit, even to the slaves themselves.[107] Defenders of slavery argued that the sudden end to the slave economy would have had a fatal economic impact in the South, and there would be widespread unemployment and chaos; slave labor was the foundation of their economy.[109] The plantations were highly profitable, due to the heavy European demand for raw cotton. Northern cities and regional industries were tied economically to slavery by banking, shipping, and manufacturing, including textile mills. In addition, southern states benefited by their increased apportionment in Congress due to the partial counting of slaves in their populations.\n The issue of slavery in the new territories was seemingly settled by the Compromise of 1850, which included the admission of California as a free state in exchange for no federal restrictions on slavery placed on Utah or New Mexico.[110] A point of contention was the Fugitive Slave Act, which required the states to cooperate with slave owners when attempting to recover escaped slaves. Formerly, an escaped slave that reached a non-slave state was presumed to have attained freedom under the Missouri Compromise.[111][112][113] The Compromise of 1820 was repealed in 1854 with the Kansas–Nebraska Act, promoted by Stephen Douglas in the name of \"popular sovereignty\" and democracy. It permitted voters to decide on the legality of slavery in each territory. Anti-slavery forces rose in anger and alarm, forming the new Republican Party. Pro- and anti- contingents rushed to Kansas to vote for or against slavery, resulting in a miniature civil war called Bleeding Kansas. By the late 1850s, the young Republican Party dominated nearly all northern states, and thus, the electoral college. It insisted that slavery would never be allowed to expand, and thus would slowly die out.[114] The Supreme Court's 1857 decision in Dred Scott v. Sandford ruled that the Compromise was unconstitutional and that free Black people were not U.S. citizens; the decision enraged Northerners. The Republicans worried the decision could be used to expand slavery.[111][112][113] After Abraham Lincoln won the 1860 election, seven Southern states seceded from the Union and formed the Confederate States of America (Confederacy) on February 8, 1861.[115] The Civil War began on April 12, 1861, when Confederate forces attacked a U.S. military installation at Fort Sumter in South Carolina. In response, Lincoln called on the states to send troops to recapture forts, protect Washington D.C., and \"preserve the Union\".[116] Lincoln's call led to four more states seceding and joining the Confederacy. A few of the (northernmost) slave states did not secede and became known as the border states; these were Delaware, Maryland, Kentucky, and Missouri.[citation needed] During the war, the northwestern portion of Virginia seceded from the Confederacy, becoming the new Union state of West Virginia.[117] The two armies' first major battle was the First Battle of Bull Run, which proved to both sides that the war would be much longer than anticipated.[116] In the western theater, the Union Army was relatively successful, with major battles such as Perryville and Shiloh, along with Union Navy gunboat dominance of navigable rivers producing strategic Union victories and destroying major Confederate operations.[118] Warfare in the eastern theater began poorly for the Union. U.S. General George B. McClellan failed to capture the Confederate capital of Richmond, Virginia, in his Peninsula campaign and retreated after attacks from Confederate General Robert E. Lee.[119] Meanwhile, in 1861 and 1862, both sides concentrated on raising and training new armies. The Union successfully gained control of the border states, driving the Confederates out.[120] Lee's Army of Northern Virginia won battles in late 1862 and spring 1863, but he pushed too hard and ignored the Union threat in the west. He invaded Pennsylvania in search of supplies and to cause war-weariness in the North.[121] The Emancipation Proclamation, issued by Lincoln on January 1, 1863, freed 3 million slaves in designated areas of the Confederacy.[122] In perhaps the turning point of the war, Lee's army was badly beaten by the Army of the Potomac at the July 1863 Battle of Gettysburg, and barely made it back to Virginia.[120] Survivors of the battle were immediately redeployed to suppress the New York City draft riots by Irish Americans protesting Civil War conscription and the city's free Black population.[121] In July 1863, Union forces under General Ulysses S. Grant gained control of the Mississippi River at the Battle of Vicksburg, splitting the Confederacy. In 1864, Union General William Tecumseh Sherman marched south from Chattanooga to capture Atlanta, a decisive victory that ended war jitters among Republicans in the North and helped Lincoln win re-election. Sherman's march to the sea was almost unopposed. Much of the South was destroyed, and could no longer provide desperately needed supplies to its armies. In spring 1864, Grant launched a war of attrition and pursued Lee to the final Appomattox campaign, which resulted in Lee surrendering in April 1865.[citation needed] By June 1865, the Union Army controlled all of the Confederacy and liberated all of the designated slaves.[122]\nThe Civil War was the world's earliest industrial war. Railroads, the telegraph, steamships, and mass-produced weapons were employed extensively. Civilian factories, mines, shipyards, and were mobilized.[123] Foreign trade increased, with the U.S. providing both food and cotton to Britain, and Britain sending in manufactured products and thousands of volunteers for the Union Army (and a few to the Confederate army). The Union blockade shut down Confederate ports. It remains the deadliest war in American history, resulting in the deaths of about 750,000 soldiers and an undetermined number of civilian casualties.[d] About ten percent of all Northern males 20–45 years old, and thirty percent of all Southern white males aged 18–40 died.[123] Many Black people died after being dislocated during the war and Reconstruction.[126] Reconstruction lasted from the end of the war until 1877.[116][127][128] Lincoln supported the Ten Percent Plan for states' re-admission, and the right of Black people to vote.[129] Lincoln was assassinated in April 1865 by John Wilkes Booth, and succeeded by Andrew Johnson.[130] After the war, the far west was developed and settled, first by wagon trains and riverboats, and then by the first transcontinental railroad. Many Northern European immigrants took up low-cost or free farms in the Prairie States. Mining for silver and copper encouraged development.[131] The severe threats of starvation and displacement of the unemployed Freedmen were met by the first major federal relief agency, the Freedmen's Bureau, operated by the Army.[132] The bureau also took in freed slaves.[citation needed] Three \"Reconstruction Amendments\" expanded civil rights for black Americans: the 1865 Thirteenth Amendment outlawed slavery;[133] the 1868 Fourteenth Amendment guaranteed equal rights and citizenship for Black people;[134] the 1870 Fifteenth Amendment prevented race from being used to disenfranchise men.[135]\nEx-Confederates remained in control of most Southern states for over two years, until the Radical Republicans gained control of Congress in the 1866 elections. Johnson, who sought good treatment for ex-Confederates, was virtually powerless in the face of Congress; he was impeached, but the Senate's attempt to remove him from office failed by one vote. Congress enfranchised black men and temporarily banned many ex-Confederate leaders from holding office. New Republican governments came to power based on a coalition of Freedmen made up of Carpetbaggers (new arrivals from the North), and Scalawags (native white Southerners), backed by the Army. Opponents said they were corrupt and violated the rights of whites.[136] State by state, the New Republicans lost power to a conservative-Democratic coalition, which gained control of the South by 1877. In response to Radical Reconstruction, the Ku Klux Klan (KKK) emerged in 1867 as a white-supremacist organization opposed to black civil rights and Republican rule. President Ulysses Grant's enforcement of the Ku Klux Klan Act of 1870 shut them down.[136] Paramilitary groups, such as the White League and Red Shirts emerging around 1874, openly intimidated and attacked Black people voting.[136] Reconstruction ended after the disputed 1876 election. The Compromise of 1877 gave Republican Rutherford B. Hayes the presidency in exchange for removing all remaining federal troops in the South.[137] In 1882, the United States passed the Chinese Exclusion Act (which barred all Chinese immigrants except for students and businessmen),[138] and the Immigration Act of 1882 (which barred all immigrants with mental health issues).[139] From 1890 to 1908, southern states effectively disenfranchised Black and poor white voters by making voter registration more difficult through poll taxes and literacy tests. Black people were segregated from whites in the violently-enforced Jim Crow system.[140][141] The \"Gilded Age\" was a term that Mark Twain used to describe the period of the late 19th century with a dramatic expansion of American wealth and prosperity, underscored by mass corruption in government.[142] Some historians have argued that the United States was effectively plutocratic for at least part of the era.[143][144][145] As financiers and industrialists such as J.P. Morgan and John D. Rockefeller began to amass vast fortunes, many observers were concerned that the nation was losing its pioneering egalitarian spirit.[146] An unprecedented wave of immigration from Europe served to both provide the labor for American industry and create diverse communities in previously undeveloped areas. From 1880 to 1914, peak years of immigration, more than 22 million people migrated to the country.[147] Most were unskilled workers who quickly found jobs in mines, mills, and factories. Many immigrants were craftsmen and farmers who purchased inexpensive land on the prairies. Poverty, growing inequality and dangerous working conditions, along with socialist and anarchist ideas diffusing from European immigrants, led to the rise of the labor movement.[148][149][150] Dissatisfaction on the part of the growing middle class with the corruption and inefficiency of politics, and the failure to deal with increasingly important urban and industrial problems, led to the dynamic progressive movement starting in the 1890s. Progressives called for the modernization and reform of decrepit institutions in the fields of politics, education, medicine, and industry.[151]\"Muckraking\" journalists exposed corruption in business and government, and highlighted rampant inner-city poverty. Progressives implemented antitrust laws and regulated such industries of meat-packing, drugs, and railroads. Four new constitutional amendments – the Sixteenth through Nineteenth – resulted from progressive activism, bringing the federal income tax, direct election of Senators, prohibition, and female suffrage.[151] In 1881, President James A. Garfield was assassinated by Charles Guiteau.[152] Skilled workers banded together to control their crafts and raise wages by forming labor unions in industrial areas of the Northeast. Samuel Gompers led the American Federation of Labor (1886–1924), coordinating multiple unions. In response to heavy debts and decreasing farm prices, wheat and cotton farmers joined the Populist Party.[153] The Panic of 1893 created a severe nationwide depression.[154] Many railroads went bankrupt. Labor unrest involved numerous strikes, most notably the violent Pullman Strike of 1894, which was forcibly shut down by federal troops. One of the disillusioned leaders of the Pullman strike, Eugene V. Debs, went on to become the leader of the Socialist Party of America.[155] Important legislation of the era included the 1883 Civil Service Act, which mandated a competitive examination for applicants for government jobs, the 1887 Interstate Commerce Act, which ended railroads' discrimination against small shippers, and the 1890 Sherman Antitrust Act, which outlawed monopolies in business.[142] After 1893, the Populist Party gained strength among farmers and coal miners, but was overtaken by the even more popular Free silver movement, which demanded using silver to enlarge the money supply and end the depression.[156] Financial and railroad communities fought back hard, arguing that only the gold standard would save the economy. In the 1896 presidential election, conservative Republican William McKinley defeated silverite William Jennings Bryan.[157] Prosperity returned under McKinley. The gold standard was enacted, and the tariff was raised. By 1900, the U.S. had the strongest economy in the world.[158] McKinley was assassinated by Leon Czolgosz in 1901, and was succeeded by Theodore Roosevelt.[159] The period also saw a major transformation of the banking system, with the arrival of the first credit union in 1908 and the creation of the Federal Reserve System in 1913.[160][161] Apart from two short recessions in 1907 and 1920, the economy remained prosperous and growing until 1929.[158] The United States Army continued to fight wars with Native Americans as settlers encroached on their traditional lands. Gradually the U.S. purchased tribal lands and extinguished their claims, forcing most tribes onto subsidized reservations. According to the U.S. Census Bureau in 1894, from 1789 to 1894, the Indian Wars killed 19,000 white people and more than 30,000 Indians.[163] The Spanish–American War began when Spain refused American demands to reform its oppressive policies in Cuba.[164] The war was a series of quick American victories on land and at sea. At the Treaty of Paris peace conference the United States acquired the Philippines, Puerto Rico, and Guam.[165] Cuba became an independent country, under close American tutelage. William Jennings Bryan led his Democratic Party in opposition to control of the Philippines, which he denounced as imperialism.[165] After defeating an insurrection by Filipino nationalists, the United States achieved little in the Philippines except in education. Infrastructural development lost much of its early vigor with the failure of the railroads.[166] By 1908, however, Americans lost interest in an empire and turned their international attention to the Caribbean, especially the building of the Panama Canal. The canal opened in 1914 and increased trade with Japan and the rest of the Far East. A key innovation was the Open Door Policy, whereby the imperial powers were given equal access to Chinese business, with none of them allowed to take control of China.[167] The women's suffrage movement reorganized after the Civil War. By the end of the century, a few Western states had granted women full voting rights,[106] and women gained rights in areas such as property and child custody law.[168] Around 1912, the feminist movement reawakened, putting an emphasis on its demands for equality, and arguing that the corruption of American politics demanded purification by women.[169]Alice Paul split from the large, moderate National American Woman Suffrage Association (NAWSA) led by Carrie Chapman Catt, and formed the more militant National Woman's Party. Suffragists were arrested during their \"Silent Sentinels\" pickets at the White House, and were taken as political prisoners.[170] The anti-suffragist argument that only men could fight in a war, and therefore only men deserve the right to vote, was refuted by the participation of American women on the home front in World War I. The success of woman's suffrage was demonstrated by the politics of states which already allowed women to vote, including Montana, who elected the first woman to the House of Representatives, Jeannette Rankin. The main resistance came from the South, where white leaders were worried about the threat of Black women voting. Congress passed the Nineteenth Amendment in 1919, and women could vote in 1920.[172] Politicians responded to the new electorate by emphasizing issues of special interest to women, especially prohibition, child health, and world peace.[173][174] As World War I raged in Europe from 1914, President Woodrow Wilson declared neutrality, but warned Germany that resumption of unrestricted submarine warfare against American ships would mean war. Germany decided to take the risk, and try to win by cutting off supplies to Britain through the sinking of ships such as the RMS Lusitania. The U.S. declared war in April 1917.[175] By the summer of 1918 soldiers in General John J. Pershing's American Expeditionary Forces arrived at the rate of 10,000 a day, while Germany was unable to replace its losses.[176] Dissent against the war was suppressed by the Sedition Act of 1918 and Espionage Act of 1917. Over 2,000 were imprisoned for speaking out against the war.[177] The Allies won in November 1918. Wilson dominated the 1919 Paris Peace Conference, putting his geopolitical hopes in the new League of Nations as Germany was treated harshly in the Treaty of Versailles (1919). Wilson refused to compromise with Senate Republicans over the issue of Congressional power to declare war, and the Senate rejected the Treaty and the League.[178] Instead, the United States chose to pursue unilateralism.[179] The aftershock of Russia's October Revolution resulted in fears of Communism in the United States, leading to a Red Scare and the deportation of non-citizens considered subversive.\n Despite the Progressive-era modernization of hospitals and medical schools,[180] the country lost around 550,000 lives to the Spanish flu pandemic in 1918 and 1919.[181][182] During the \"Roaring\" 1920s, the economy expanded. African-Americans benefited from the Great Migration and had more cultural power, an example being the Harlem Renaissance which spread jazz music. Meanwhile, the Ku Klux Klan had a resurgence, and the Immigration Act of 1924 was passed to strictly limit the number of new entries.[183] Prohibition began in 1920, when the manufacture, sale, import and export of alcohol were prohibited by the Eighteenth Amendment. Bootlegged alcohol in the cities ended up under the control of gangs, who fought each other for territory. Italian bootleggers in New York City gradually formed the Mafia crime syndicate. In 1933, President Franklin D. Roosevelt repealed prohibition.[184] The Great Depression (1929–1939) and the New Deal (1933–1936) were decisive moments in American political, economic, and social history.[185] A financial bubble was fueled by an inflated stock market, which led to the Wall Street crash on October 29, 1929.[186] This, along with other economic factors, triggered a worldwide depression. The United States experienced deflation as prices fell, unemployment soared from 3% in 1929 to 25% in 1933, farm prices fell by half, and manufacturing output plunged by one-third.\n The New Deal enacted by President Franklin D. Roosevelt was a series of permanent reform programs including Social Security, unemployment relief and insurance, public housing, bankruptcy insurance, farm subsidies, and regulation of financial securities.[187] It also provided unemployment relief through the Works Progress Administration (WPA) and for young men, the Civilian Conservation Corps. Large-scale spending projects designed to rebuild infrastructure were under the purview of the Public Works Administration.[187] State governments introduced the sales tax to pay for new programs. Ideologically, the New Deal established modern liberalism in the United States.[187] The New Deal coalition won re-election for Roosevelt in 1936, 1940, and 1944.[187] The Second New Deal in 1935 and 1936 brought the economy further left, building up labor unions through the Wagner Act. Roosevelt weakened his second term by a failed effort to pack the Supreme Court, which had been a center of conservative resistance to his programs.[187] The economy essentially recovered by 1936, but long-term unemployment remained a problem until it was solved by wartime spending. Most of the relief programs were dropped in the 1940s, when the conservatives regained power in Congress through the Conservative coalition.[187] During the Depression, the United States remained focused on domestic concerns. U.S. legislation in the Neutrality Acts sought to avoid foreign conflicts; however, policy clashed with increasing anti-Nazi feelings following the German invasion of Poland in September 1939 that started World War II.[188] At first, Roosevelt positioned the U.S. as the \"Arsenal of Democracy\", pledging full-scale financial and munitions support for the Allies and Lend-Lease agreements – but no military personnel.[188] Japan tried to neutralize America's power in the Pacific by attacking Pearl Harbor in 1941, but instead it catalyzed American support to enter the war.[189] Roosevelt's Executive Order 9066 resulted in over 120,000 Americans of Japanese descent being removed from their homes and placed in internment camps.[190][191][192] The Allies fought against Germany in the European theater and Japan in the Pacific War.[193] The United States was one of the \"Allied Big Four\", alongside the United Kingdom, Soviet Union, and China.[194][195] The U.S. gave the Allied war effort money, food, petroleum, technology, and military personnel. The U.S. focused on maximizing its national economic output, causing a dramatic increase in GDP, the end of unemployment, and a rise in civilian consumption, even as 40% of the GDP went to the war effort.[185] A wartime production boom led to the end of the Great Depression. Tens of millions of workers moved into the active labor force and to higher-productivity jobs. Labor shortages encouraged industries to look for new sources of workers, finding new roles for women and Black people. Economic mobilization was managed by the War Production Board.[185] Most durable goods became unavailable or were tightly rationed, while housing for industrial jobs was in short supply. Prices and wages were controlled, and Americans saved a high portion of their incomes, which led to post-war growth.[196][197] The U.S. stopped Japanese expansion in the Pacific in 1942; after the loss of the Philippines to Japanese conquests, as well as a draw in the Battle of the Coral Sea in May, the American Navy then inflicted a decisive blow at Midway in June 1942. The Allied forces built up a garrison on Guadalcanal island, formerly held by the Japanese, after the successes of the Battle of the Eastern Solomons and the Battle of Guadalcanal. The Japanese then stopped advancing south, and the U.S. began taking New Guinea. Japan also lost their invasion of the Alaskan Aleutian Islands, allowing the U.S. to begin attacking the Japanese-controlled Kuril Islands.[193] American ground forces assisted in the North African campaign and the collapse of Fascist Italy in 1943. A more significant European front was opened on D-Day, June 6, 1944, in which Allied forces invaded Nazi-occupied France.[193] The Allies began pushing the Germans out of France in the Normandy campaign. After Allied forces landed at the French Riviera in Operation Dragoon, Hitler allowed his army to retreat from Normandy.[198] Roosevelt died in 1945, and was succeeded by Harry Truman.[199] The western front stopped short of Berlin, leaving the Soviets to take it in the Battle of Berlin. The Nazi regime formally capitulated in May 1945, ending the war in Europe.[193] In the Pacific, the U.S. implemented an island hopping strategy toward Tokyo. The Philippines was eventually reconquered, after Japan and the United States fought in history's largest naval battle, the Battle of Leyte Gulf.[200] After the war, the U.S. granted the Philippines independence.[201] Military research and development increased during the war, leading to the Manhattan Project, a secret effort to harness nuclear fission to produce atomic bombs;[202] the first nuclear device was detonated on July 16, 1945.[203] U.S. airfields in the Mariana Islands allowed for easier bombing of Japan and hard-fought U.S. victories at Iwo Jima and Okinawa in 1945.[204] The U.S. prepared to invade Japan's home islands, but they dropped atomic bombs on the Japanese cities of Hiroshima and Nagasaki, compelling Japan to surrender and ending World War II.[205] The U.S. occupied Japan (and part of Germany).[206] 400,000 American military personnel and civilians were killed.[207] Nuclear weapons have not been used since the war ended, and a \"long peace\" began between the global powers, but they still competed in the Cold War.[208] In the decades after World War II, the United States became a global influence in economic, political, military, cultural, and technological affairs. Following World War II, the United States emerged as one of the two dominant superpowers, the Soviet Union being the other. The U.S. Senate approved U.S. participation in the United Nations (UN), which marked a turn away from traditional American isolationism and toward increased international involvement.[209] The United States and other major Allied powers became the foundation of the UN Security Council.[210] The Central Intelligence Agency (CIA) was created in 1947.[211] America wished to rescue Europe from the devastation of World War II, and to contain the expansion of communism, represented by the Soviet Union. U.S. foreign policy during the Cold War was built around the support of Western Europe and Japan along with the policy of containment (containing the spread of communism).[212] The Truman Doctrine in 1947 was the U.S.' attempt to secure trading partners in Europe, by providing military and economic aid to Greece and Turkey to counteract the threat of communist expansion in the Balkans.[213][208] In 1948, the United States replaced piecemeal financial aid programs with a comprehensive Marshall Plan, which pumped money into Western Europe, and removed trade barriers, while modernizing the managerial practices of businesses and governments.[214] Post-war American aid to Europe totaled $25 billion, out of the U.S. GDP of $258 billion in 1948.[214] In 1949, the United States, rejecting the long-standing policy of no military alliances in peacetime, formed the North Atlantic Treaty Organization (NATO) alliance. In response, the Soviets formed the Warsaw Pact of communist states, leading to the \"Iron Curtain\".[214] In 1949, the Soviets performed their first nuclear weapon test.[208] This escalated the risk of nuclear warfare; the threat of mutually assured destruction, however, prevented both powers from nuclear war, and resulted the proxy wars in which the two sides did not directly confront each other.[208] The U.S. fought against communists in the Korean War and Vietnam War, and toppled left-wing governments in the third world to try to stop its spread, such as Iran in 1953 and Guatemala in 1954.[212]McCarthyism was a widespread government program led by Senator Joseph McCarthy to expose communists in government and business. Hollywood was targeted by the House Un-American Activities Committee.[215] Gay people were targeted under the McCarthyist Lavender Scare.[216] Dwight D. Eisenhower was elected president in 1952 in a landslide.[217] He ended the Korean War, and avoided any other major conflict. He cut military spending by relying on advanced technology, such as nuclear weapons carried by long-range bombers and intercontinental missiles.[218] After Stalin died in 1953, Eisenhower worked to obtain friendlier relationships with the Soviet Union. At home, he ended McCarthyism, expanded the Social Security program, and presided over a decade of bipartisan cooperation.[218] Domestically, after 1948, America entered an economic boom: 60% of the American population had attained a \"middle-class\" standard of living by the mid-1950s, compared with only 31% in the 1928 and 1929. Between 1947 and 1960, the average real income for American workers increased by as much as it had in the previous half-century.[219] The economy allowed for an affordable lifestyle with large families; this created the baby boom, in which millions of children were born at increased rates from 1945 to 1964.[220] Many Americans moved into suburban neighborhoods.[221] In the 1954 Supreme Court decision Brown v. Board of Education, public school segregation was ruled unconstitutional.[222] When nine Black students were threatened over their admission into all-white Little Rock Central High School, Eisenhower sent in a thousand National Guard troops to ensure peace.[218] Starting in the late 1950s, institutionalized racism across the United States, but especially in the South, was increasingly challenged by the growing civil rights movement. The activism of Rosa Parks and Martin Luther King Jr. led to the boycott of segregated public buses in Montgomery, Alabama in 1955, organized by King and the Montgomery Improvement Association. They faced multiple acts of violence, but continued the boycott for a year, until the Supreme Court ordered the city to desegregate the buses.[222] The Soviets unexpectedly surpassed American technology in 1957 with Sputnik, the first Earth satellite. The R-7 missile which launched Sputnik into space could have hypothetically dropped a nuclear bomb into U.S. air space from above; new American fears regarding Soviet power began the Space Race, a competition between the two countries to prove their technological superiority through space exploration. In 1958, Eisenhower created the National Aeronautics and Space Administration (NASA) for this purpose. Angst about the weaknesses of American education led to large-scale federal support for science education and research.[223] In 1960, John F. Kennedy was elected President. His administration saw the acceleration of the country's role in the Space Race, escalation of the American role in the Vietnam War, the Bay of Pigs Invasion, and the Cuban Missile Crisis. President Kennedy was assassinated on November 22, 1963.[224]Lyndon B. Johnson then became president.[225] He secured congressional passage of his Great Society programs,[226] dealing with civil rights, the end of legal segregation, Medicare, extension of welfare, federal aid to education at all levels, subsidies for the arts and humanities, environmental activism, and a series of programs designed to wipe out poverty.[227][228] For years, nonviolent civil rights activists organized direct actions, such as the 1963 Birmingham campaign and 1965 Selma to Montgomery march, where they also became victims of violence. Along with Supreme Court decisions like Loving v. Virginia and the 1963 March on Washington, these movements achieved great steps toward equality with laws like the Civil Rights Act of 1964, the Voting Rights Act of 1965, and the Fair Housing Act of 1968. These ended the Jim Crow laws that had legalized racial segregation.[229] Native Americans protested federal courts, highlighting the federal government's failure to honor treaties involving them. One of the most outspoken Native American groups was the American Indian Movement (AIM). In the 1960s, Cesar Chavez began organizing poorly paid Mexican-American farm workers in California, eventually forming the country's first successful union of farm workers, the United Farm Workers of America (UFW).[230] Amid the Cold War, the United States entered the Vietnam War, whose growing unpopularity fed already existing social movements. Feminism and the environmental movement became political forces, and progress continued toward civil rights for all Americans. A counterculture revolution in the late sixties and early seventies further divided Americans in a \"culture war\" but also brought forth more liberated social views.[231] Frustrations with the seemingly slow progress of the integration movement led to the emergence of more radical politics, such as the Black Power movement.[232] The summer of 1967 saw opposing philosophies in two widespread movements, the more peaceful summer of love and the radical long, hot summer, which included nationwide riots.[233] Martin Luther King Jr. was assassinated in 1968.[234] The modern gay rights movement started after the Stonewall riots in 1969.[235] A new consciousness of the inequality of American women began sweeping the nation, starting with the 1963 publication of Betty Friedan's best-seller, The Feminine Mystique, which critiqued the American cultural idea that women could only find fulfillment through their roles as wives, mothers, and keepers of the home. In 1966, Friedan and others established the National Organization for Women (NOW) to advocate for women's rights.[168][236] Protests began, and the new women's liberation movement grew in size and power, gaining much media attention.[237] The proposed Equal Rights Amendment to the Constitution, passed by Congress in 1972, was defeated by a conservative coalition mobilized by Phyllis Schlafly.[237][238] However, many federal laws established women's equal status under the law, such as those equalizing pay, employment, education, employment opportunities, and credit between genders, and ending pregnancy discrimination. State laws criminalized spousal abuse and marital rape, and the Supreme Court ruled that the equal protection clause of the Fourteenth Amendment applied to women. Social custom and consciousness began to change, accepting women's equality. Abortion, deemed by the Supreme Court as a fundamental right in Roe v. Wade (1973), is still a point of debate.\n President Richard Nixon (1969–1974) largely continued the New Deal and Great Society programs he inherited.[225][239] Nixon created the Environmental Protection Agency,[240]opened relations with China, and attempted to gradually turn the Vietnam War effort over to the South Vietnamese. He negotiated the peace treaty in 1973 which secured the release of POWs and led to the withdrawal of U.S. troops. The war had cost the lives of 58,000 American troops. Nixon manipulated the fierce distrust between the Soviet Union and China to the advantage of the U.S., achieving détente with both parties.[239] He was also president during the U.S.' landing on the Moon in 1969.\n The Watergate scandal, involving Nixon's cover-up of his operatives' break-in into the Democratic National Committee headquarters, destroyed his political base and forced his resignation on August 9, 1974.[239] He was succeeded by Vice President Gerald Ford.[239][241] The Fall of Saigon on April 30, 1975, ended the Vietnam War.[239] In Central America, the U.S. government supported right-wing governments against left-wing groups, such as in El Salvador and Guatemala. In South America, they supported Argentina and Chile, who carried out Operation Condor, a campaign of assassinations of exiled political opponents by Southern Cone governments, created at the behest of Chilean dictator Augusto Pinochet in 1975.[242][243][244] The OPEC oil embargo marked a long-term economic transition: energy prices skyrocketed, and American factories faced serious competition from foreign automobiles, clothing, electronics, and consumer goods. By the late 1970s, the economy suffered an energy crisis, slow economic growth, high unemployment, very high inflation, and high interest rates (stagflation). Since economists agreed on deregulation, many of the New Deal era regulations were ended.[245] Meanwhile, the first mass-market personal computers were being developed in California's Silicon Valley.[246] Jimmy Carter was elected president in 1976.[247] Carter brokered the Camp David Accords between Israel and Egypt. In 1979, Iranian students stormed the U.S. embassy in Tehran and took 66 Americans hostage. Carter lost the 1980 election to the Republican Ronald Reagan.[248] On January 20, 1981, minutes after Carter's term ended, the remaining U.S. captives were released.\n President Ronald Reagan's conservative policies produced a major political realignment with his 1980 and 1984 landslide elections.[249][250] Reagan's neoliberal economic policies (dubbed \"Reaganomics\") included the implementation of the Economic Recovery Tax Act of 1981.[250][251][252] Reagan continued to downsize government taxation and regulation;[253] New Deal and Great Society programs were ended.[225] The U.S. experienced a recession in 1982, but after inflation decreased, unemployment then decreased, and the economic growth rate increased from 4.5% in 1982 to 7.2% in 1984.[254][255] However, homelessness and economic inequality also rose.[256][257] The Reagan administration's expansion of the War on Drugs led to an increase in incarceration, particularly among African Americans, with the number of people imprisoned for drug offences rising from 50,000 to 400,000 between 1980 and 1997.[258][259] Manufacturing industries moving out of inner cities increased poverty in those areas; poverty increased drug dealing and contributed to the crack epidemic, which led to increased crime and incarceration.[258][260] The government also reacted slowly to the AIDS crisis, and banned reliable information on the disease, which led to higher infection rates.[261][262] Reagan ordered a buildup of the U.S. military, incurring additional budget deficits.[263] The 1983 invasion of Grenada and 1986 bombing of Libya were popular in the U.S., though Reagan's backing of the Contra rebels was mired in the controversy over the Iran–Contra affair.[264] Reagan also introduced a complicated missile defense system known as the Strategic Defense Initiative. The Soviets reacted harshly because they thought it violated the 1972 Anti-Ballistic Missile Treaty, and would give the U.S. a major military advantage, so they stopped negotiating disarmament deals until the late 1980s.[263] Reagan met four times with Gorbachev, and their summit conferences led to the signing of the Intermediate-Range Nuclear Forces Treaty. The Soviet Union collapsed in 1991, ending the Cold War.[265] For the remainder of the 20th century, the United States emerged as the world's sole remaining superpower and continued to intervene in international affairs during the 1990s.[266] The Gulf War against Iraq started during George H.W. Bush's administration.[266] The war started when Iraq invaded neighboring Kuwait in 1990. After an international U.S.-led offensive, Kuwait was taken back. Under Bush, the U.S. also became involved in wars in Panama, Somalia, Bosnia, and Croatia.[267][268][269] In 1992, there were riots in Los Angeles over police brutality.[270] Elected in 1992, President Bill Clinton oversaw economic expansion and passed the first balanced federal budget in 30 years.[266] Much of the economic boom was a side effect of the Digital Revolution, and new business opportunities created by the Internet.[271] During Clinton's administration, the U.S. was involved in wars in Haiti and Kosovo.[272][273] Conservative Republicans heavily won the 1994 midterm elections in a \"Republican Revolution\" which was built around the Contract with America policy agenda.[274][275]Newt Gingrich was chosen as House Speaker,[274] and he would heavily influence the Republican Party to engage in \"confrontational\" political speech.[276][277] Clinton's leadership after the Oklahoma City bombing increased his popularity,[278] and he won in the 1996 presidential elections.[279] In 1998, Clinton was impeached by the House of Representatives on charges of lying under oath about a sexual relationship with White House intern Monica Lewinsky. He was acquitted by the Senate.[266] In 2000, the dot-com bubble, a widespread overvaluation of Internet company stocks, burst and hurt the U.S. economy.[280][281] The close presidential election in 2000 between Governor George W. Bush and Al Gore was extremely close and produced a dramatic dispute over the counting of votes.[282] Bush ultimately won.[283] On September 11, 2001, 19 al-Qaeda hijackers commandeered four commercial planes to be used in suicide attacks. Two were crashed intentionally into both Twin Towers of the World Trade Center in New York City, and a third into the Pentagon in Arlington, Virginia. The fourth plane was re-taken by the passengers and crew and crashed into an empty field. Every building of the World Trade Center partially or completely collapsed, massively damaging the surrounding area and blanketing Lower Manhattan in toxic dust clouds. 2,977 victims died in the attacks. On September 20, Bush announced a \"war on terror\".[284][285] In October 2001, the U.S. and NATO invaded Afghanistan and ousted the Taliban regime, which had harbored al-Qaeda and its leader Osama bin Laden.[286] Bin Laden escaped to Pakistan, starting a manhunt.[287] The U.S. established new domestic efforts to prevent future attacks. The Patriot Act increased the power of government to monitor communications and removed legal restrictions on intelligence sharing between federal law enforcement agencies.[288] The government's indefinite detention of terrorism suspects captured abroad at the Guantanamo Bay detention camp led to allegations of human rights abuses and violations of international law.[289][290][291] The Department of Homeland Security was created to lead federal counter-terrorism activities.[288] In March 2003, the U.S. launched an invasion of Iraq, claiming Iraqi dictator Saddam Hussein had weapons of mass destruction (WMDs). Intelligence backing WMDs were later found to be inaccurate. The war led to the collapse of the Iraqi government and the eventual capture of Hussein.[292][293] The Iraq War fueled international protests and gradually saw domestic support decline.[294][295] In 2005, Hurricane Katrina killed 1,800 people around New Orleans after the city's levees broke.[296] In 2007, after years of violence by the Iraqi insurgency, Bush deployed more troops in a strategy dubbed \"the surge\". While the death toll decreased, the political stability of Iraq remained in doubt.[297] In 2008, the U.S. entered the Great Recession.[298][299] Multiple overlapping crises were involved, especially the housing market crisis, a subprime mortgage crisis, soaring oil prices, an automotive industry crisis, rising unemployment, and the worst financial crisis since the Great Depression. The financial crisis threatened the stability of the entire economy in September 2008, when Lehman Brothers failed, and other giant banks were in grave danger.[300] Starting in October, the federal government lent $245 billion to financial institutions through the bipartisan Troubled Asset Relief Program.[301][302] Barack Obama, the first multiracial[303] president, was elected in 2008.[304] He signed the Don't Ask, Don't Tell Repeal Act, which allowed people to serve in the military while openly gay.[305] To help the economy, he signed the American Recovery and Reinvestment Act of 2009,[306]Consumer Assistance to Recycle and Save Act,[307] the Patient Protection and Affordable Care Act (informally \"Obamacare\"),[308] and the Dodd–Frank Wall Street Reform and Consumer Protection Act.[309][310] The unemployment rate began falling as the economy and labor markets experienced a recovery.[311] These changes to the economic system created new political movements, such as the Occupy movement and the Tea Party movement.[312] The recession officially ended in mid-2009.[313] Following the 2010 midterm elections, Congress was in gridlock,[314] leading to the Budget Control Act of 2011.[315] The economic expansion that followed the Great Recession was the longest in U.S. history;[316][317] the unemployment rate reached a 50-year low in 2019.[318] Despite the strong economy, increases in the cost of living surpassed increases in wages.[319][320]\nIn 2009, Obama issued an executive order banning the use of torture.[321][322] He ordered the closure of secret CIA-run prisons overseas,[323][324] and sought to close the Guantanamo Bay detention camp, but his efforts were stymied by Congress.[325][326] American military personnel left Iraq in 2011.[327] Meanwhile, Obama increased involvement in Afghanistan, adding 30,000 troops, while proposing to begin withdrawal in 2014.[322] The U.S., with NATO, intervened in the Libyan Civil War in 2011.[328] In May 2011, Osama bin Laden was killed in Pakistan in a Navy SEALs raid ordered by Obama. While Al Qaeda was near collapse in Afghanistan, affiliated organizations continued to operate in Yemen and other remote areas, as the CIA used drones to hunt down its leadership.[329][330] In October, Obama sent troops to Central Africa to fight the Lord's Resistance Army.[331] \nFollowing Obama's 2012 re-election, Congressional gridlock continued, resulting in the first government shutdown since the Clinton administration.[332] In 2012, the Sandy Hook Elementary School shooting in Newtown, Connecticut, led to unsuccessful attempts from Obama to promote gun reform.[333] The Boston Marathon bombing of 2013 killed three people and injured more than 260.[334] In 2013, the U.S. also started a counter-terrorist invention in Niger,[335] and began a covert operation to train rebels in Syria who were fighting against the terrorist group ISIS. The latter program was publicized and expanded in 2014.[336] That year, ISIS grew in scope in the Middle East, and inspired many terrorist attacks in the United States, including the 2015 San Bernardino attack.[337][338][339] The U.S. and its allies began a significant military offensive against ISIS in Iraq which lasted from 2014 to 2021.[340][341] In December 2014, Obama officially ended the combat mission in Afghanistan.[342] The shooting of Black teen Michael Brown by police officer Darren Wilson, and a grand jury declining to charge Wilson with murder, led to the Ferguson unrest in Missouri in 2014 and 2015.[343] In 2012, President Obama became the first president to openly support same-sex marriage.[344] The Supreme Court provided federal recognition of same-sex marriages in 2013,[345] and then legalized gay marriage nationwide with Obergefell v. Hodges in 2015.[346] Also in 2015, the U.S. joined the international Paris Agreement on climate change.[347] In November 2016, Donald Trump was elected president.[348] The election's legitimacy was disputed when the FBI and Congress investigated if Russia interfered in the election to help Trump win. There were also accusations of collusion between Trump's campaign and Russian officials. The Mueller report concluded that Russia attempted to help Trump's campaign, but there was no evidence of \"explicit\" collusion.[349][350][351] During Trump's presidency, he espoused an \"America First\" ideology, placing restrictions on asylum seekers, expanding the wall on the U.S.-Mexico border, and banning immigration from seven Muslim-majority countries. Many of his actions were challenged in court.[352][353][354] He confirmed three new Supreme Court justices (cementing a conservative majority),[355] started a trade war with China,[356] signed the Tax Cuts and Jobs Act, and removed the U.S. from the Paris Agreement.[347] In 2018, the administration separated families which were illegally immigrating to the country. After public outcry, Trump rescinded the policy.[357][358] A whistleblower complaint alleged that Trump had withheld foreign aid from Ukraine under the demand that they investigate the business dealings of Hunter Biden; Hunter's father, Democrat Joe Biden, would be Trump's opponent in the 2020 presidential election.[359][360] Trump was impeached for abuse of power and obstruction of congress, but was acquitted in 2020.[361] In the 2010s and early 2020s, Americans became more politically polarized.[362][363][364] The #MeToo movement exposed alleged sexual harassment and abuse in the workplace.[365] Many celebrities were accused of misconduct or rape.[366][367] The Black Lives Matter movement gained support after multiple police killings of African-Americans.[368] White supremacy also grew.[369][370][371] The 2017 Women's March against Trump's presidency was one of the largest protests in American history.[372] Multiple mass shootings, including the 2016 Pulse Nightclub shooting, 2017 Las Vegas shooting, and 2018 Parkland shooting, led to increased calls for gun reform, such as in the March for Our Lives student protest movement.[373][374] COVID-19 started spreading in China in 2019.[375] In March 2020, the WHO declared a pandemic.[376] American state and local governments imposed stay-at-home orders to slow the virus' spread, reducing patient overload in hospitals. By April, the U.S. had the most cases of any country, at 100,000.[377][378][379] On April 11, the U.S. death toll became the highest in the world at 20,000,[380] and by May 2022, one million had died.[381] Unemployment rates were the highest since the Great Depression.[382][383] The biggest mass vaccination campaign in U.S. history started in December 2020.[384] \nThe May 2020 murder of George Floyd caused mass protests and riots in many cities over police brutality.[385] Many organizations attempted to rid themselves of institutionalized racism.[386] 2020 was also marked by a rise in domestic terrorist threats and widespread conspiracy theories around mail-in voting and COVID-19.[387][388][389] The QAnon conspiracy theory gained publicity.[390][391] Multiple major cities were hit by rioting and fighting between far-left anti-fascist groups and far-right groups like the Proud Boys.[392][393] Joe Biden defeated Trump in the 2020 election.[360] Trump repeatedly made false claims of massive voter fraud and election rigging,[394][395][396] leading to the January 6 United States Capitol attack by supporters of Trump and right-wing militias.[397][398] The attack was widely described as a coup d'état.[399][400][401] It led to Trump's impeachment for incitement of insurrection, making him the only U.S. president to be impeached twice.[402][403][404] The Senate later acquitted Trump, despite some fellow Republicans voting against him.[405][406]Kamala Harris was inaugurated as the first Black, Asian, and female vice president.[407] In 2021, Biden finished the withdrawal of American troops from Afghanistan which started under Trump. After an evacuation of over 120,000 American citizens, Afghanistan fell to the Taliban in August.[408][409][410] Biden signed into law the American Rescue Plan Act of 2021, a $1.9 trillion stimulus bill.[411] He also proposed a significant expansion of the social safety net through the Build Back Better Act, but those efforts, along with voting rights legislation, failed in Congress.[412] He signed bills regarding infrastructure,[413]gun reform,[414]inflation reduction,[415] and healthcare for veterans,[416] among other issues.[417] New preventative restrictions were put in place in reaction to the SARS-CoV-2 Delta variant.[418][419][420] In the early 2020s, Republican-led states began sweeping rollbacks of LGBT rights.[421] In 2022, the Supreme Court ruled in Dobbs v. Jackson that having an abortion is not a protected Constitutional right, overturning Roe v. Wade and Planned Parenthood v. Casey and sparking nationwide protests.[422][423] Biden appointed Ketanji Brown Jackson to become the first Black woman to serve on the court.[424] In 2023, Trump began appearing in court as a defendant in multiple notable criminal trials.[425][426] Meanwhile, the U.S. began supporting Israel in the Israel-Hamas war[427] and protecting shipping in the Red Sea from attacks by the Yemeni Houthis.[428] In June 2024, Trump became the first president convicted of a crime, when he was found guilty of 34 felony counts for falsifying business documents related to his paying off of Stormy Daniels.[429] In July, the Supreme Court ruled in Trump v. United States that presidents are somewhat immune from criminal prosecution, helping Trump before his planned election subversion trial.[430][431][432] Later in July, Biden dropped out of the 2024 race, endorsing Kamala Harris.[433] During the election season, there were two assassination attempts on Trump.[434] Trump won the 2024 presidential election.[435][436]"I want to know which years are mentioned in the text.
Write a regex that can extract all the years from the text. Don’t worry about any years before 1000, its a young country.