When she was five years old, Krauss began playing the violin, taking classical lessons. She soon tired of the regimen of classical playing and began performing country and bluegrass licks. At the age of eight, she began entering talent contests in and around her native Champaign, IL. Two years later, she had her own band. In 1983, when she was 12 years old, she won the Illinois State Fiddle Championship and the Society for the Preservation of Bluegrass in America named her the Most Promising Fiddler in the Midwest. In 1985, Krauss made her recording debut on an album, playing on a record made by her brother Viktor, Jim Hoiles, and Bruce Weiss. The album was called Different Strokes and appeared on the independent Fiddle Tunes label. Later that year, she signed to Rounder Records. She was 14 years old at the time.

Too Late to Cry, Krauss' debut album, appeared in 1987 to very positive reviews. The album was recorded with Krauss' backup band, the Union Station, which featured guitarist Jeff White, banjoist Alison Brown, and bassist Viktor Krauss; the following year, the group won the Society for the Preservation of Bluegrass in America's National Band Championship contest. In 1989, Krauss and Union Station released Two Highways, which was nominated for the Grammy Award for Best Bluegrass Recording. Although the album didn't win the award, her next album, 1990's I've Got That Old Feeling, did. The success of I've Got That Old Feeling was unprecedented for bluegrass acts in the '80s and it laid the groundwork for Krauss' breakthrough in the '90s. By this time, Union Station's lineup had more or less settled. It now featured mandolinist Adam Steffey, banjoist/guitarist Ron Block, bassist Barry Bales, and guitarist Tim Stafford; Stafford later left the group and was replaced by Dan Tyminski.

In 1992, Alison Krauss & Union Station released Every Time You Say Goodbye, which featured a typically eclectic array of material. The album appeared in the country charts and Krauss' videos were shown on Country Music Television. I Know Who Holds Tomorrow was released in 1994 and was even more successful. But it was the 1995 compilation Now That I've Found You: A Collection that made Krauss a star. The album reached number two on the country charts and -- even more remarkably -- went into the pop Top Ten and sold over a million copies. Its success confirmed her status as bluegrass' leading light in the '90s.

Krauss & Union Station followed the unexpected success of Now That I've Found You with So Long, So Wrong in spring 1997. Forget About It followed in mid-1999. A year later, Krauss & Union Station joined the likes of John Hartford, Ralph Stanley, and others for the multi-million-selling soundtrack O Brother, Where Art Thou? A North American tour showcasing some of the album's stellar musicians followed in summer 2002, allowing Krauss and her band's popularity to soar. New Favorite appeared in November and went gold within four months. A live album followed soon after, and in 2004 Krauss released Lonely Runs Both Ways. A Hundred Miles or More, a collection drawn from Krauss' Rounder albums, along with sides recorded for various soundtrack projects and five previously unreleased tracks, appeared in 2007. Also appearing in 2007 was Raising Sand, an acclaimed, multi-platinum collaboration with Robert Plant. She was back with Union Station for 2011’s Paper Airplane, which was self-produced by the band and engineered by Mike Shipley.