The prospect that thousands of drones could be patrolling U.S. skies by the end of this decade is raising the specter of a Big Brother government that peers into backyards and bedrooms.

The worries began mostly on the political margins, but there are signs that ordinary people are starting to fret that unmanned aircraft could soon be circling overhead.

Jeff Landry, a freshman Republican congressman from Louisiana's coastal bayou country, said constituents have stopped him while shopping at Walmart to talk about it.

"There is a distrust amongst the people who have come and discussed this issue with me about our government," Landry said. "It's raising an alarm with the American public."

Another GOP freshman, Rep. Austin Scott, said he first learned of the issue when someone shouted out a question about drones at a Republican Party meeting in his Georgia congressional district two months ago.

An American Civil Liberties Union lobbyist, Chris Calabrese, said that when he speaks to audiences about privacy issues generally, drones are what "everybody just perks up over."

"People are interested in the technology, they are interested in the implications and they worry about being under surveillance from the skies," he said.

The level of apprehension is especially high in the conservative blogosphere, where headlines blare "30,000 Armed Drones to be Used Against Americans" and "Government Drones Set to Spy on Farms in the United States."

When Virginia Gov. Bob McDonnell, a Republican, suggested during a radio interview last month that drones be used by police domestically since they've done such a good job on foreign battlefields, the political backlash was swift. NetRightDaily complained: "This seems like something a fascist would do. ... McDonnell isn't pro-Big Government, he is pro-HUGE Government."

John Whitehead, president of the Rutherford Institute of Charlottesville, Va., which provides legal assistance in support of civil liberties and conservative causes, warned the governor, "America is not a battlefield, and the citizens of this nation are not insurgents in need of vanquishing."

There's concern as well among liberal civil liberties advocates that government and private-sector drones will be used to gather information on Americans without their knowledge.

A lawsuit by the Electronic Frontier Foundation of San Francisco, whose motto is "defending your rights in the digital world," forced the Federal Aviation Administration earlier this year to disclose the names of dozens of public universities, police departments and other government agencies that have been awarded permission to fly drones in civilian airspace on an experimental basis.

Giving drones greater access to U.S. skies moves the nation closer to "a surveillance society in which our every move is monitored, tracked, recorded and scrutinized by the authorities," the ACLU warned last December in a report.

The anxiety has spilled over into Congress, where a bipartisan group of lawmakers have been meeting to discuss legislation that would broadly address the civil-liberty issues raised by drones.

A Landry provision in a defense spending bill would prohibit information gathered by military drones without a warrant from being used as evidence in court. A provision that Rep. Rush Holt, D-N.J., added to another bill would prohibit the Homeland Security Department from arming its drones, including ones used to patrol the border.

Scott and Sen. Rand Paul, R-Ky., have introduced identical bills to prohibit any government agency from using a drone to "gather evidence or other information pertaining to criminal conduct or conduct in violation of a regulation" without a warrant.

"I just don't like the concept of drones flying over barbecues in New York to see whether you have a Big Gulp in your backyard or whether you are separating out your recyclables according to the city mandates," Paul said in an interview, referring to a New York City ban on supersized soft drinks.

He acknowledged that is an "extreme example," but added: "They might just say we'd be safer from muggings if we had constant surveillance crisscrossing the street all the time. But then the question becomes, what about jaywalking? What about eating too many donuts? What about putting mayonnaise on your hamburger? Where does it stop?"

Calabrese, the ACLU lobbyist, called Paul's office as soon as he heard about the bill.

"I told them we think they are starting from the right place," Calabrese said. "You should need some kind of basis before you use a drone to spy on someone."

In a Congress noted for its political polarization, legislation to check drone use has the potential to forge "a left-right consensus," he said. "It bothers us for a lot of the same reasons it bothers conservatives."

The backlash has drone makers concerned. The drone market is expected to nearly double over the next 10 years, from current worldwide expenditures of nearly $6 billion annually to more than $11 billion, with police departments accounting for a significant part of that growth.

"We go into this with every expectation that the laws governing public safety and personal privacy will not be administered any differently for (drones) than they are for any other law enforcement tool," said Dan Elwell, vice president of the Aerospace Industries Association.

Discussion of the issue has been colored by exaggerated drone tales spread largely by conservative media and bloggers.

Scott said he was prompted to introduce his bill in part by news reports that the Environmental Protection Agency has been using drones to spy on cattle ranchers in Nebraska. The agency has indeed been searching for illegal dumping of waste into streams but is doing it the old-fashioned way, with piloted planes.

In another case, a forecast of 30,000 drones in U.S. skies by 2020 has been widely attributed to the FAA. But FAA spokeswoman Brie Sachse said the agency has no idea where the figure came from. It may be a mangled version of an aerospace industry forecast that there could be nearly 30,000 drones worldwide by 2018, with the United States accounting for half of them.

Fear that some drones may be armed has been fueled in part by a county sheriff's office in Texas that used a homeland security grant to buy a $300,000, 50-pound ShadowHawk helicopter drone for its SWAT team.

The drone can be equipped with a 40mm grenade launcher and a 12-gauge shotgun. Randy McDaniel, chief deputy with the Montgomery County Sheriff's Office, told The Associated Press earlier this year his office had no plans to arm the drone, but he left open the possibility the agency may decide to adapt the drone to fire tear gas canisters and rubber bullets.

Earlier this year Congress, under pressure from the Defense Department and the drone manufacturers, ordered the FAA to give drones greater access to civilian airspace by 2015. Besides the military, the mandate applies to drones operated by the private sector and civilian government agencies, including federal, state and local law enforcement.

Reps. Ed Markey, D-Mass, and Joe Barton, R-Texas, co-chairs of a congressional privacy caucus, asked the FAA in April how it plans to protect privacy as it develops regulations for integrating drones into airspace now exclusively used by aircraft with human pilots. There's been no response so far, but Acting FAA Administrator Michael Huerta will probably be asked about it when he testifies at a Senate hearing Thursday.

Even if the FAA were to establish privacy rules, it's primarily a safety agency and wouldn't have the expertise or regulatory structure to enforce them, civil liberties advocates said. But no other government agency is addressing the issue, either, they said.