So you’ve watched in frustration for months as more of your paycheck disappears down the gas tank.

While fuel prices have dropped somewhat, for many Mainers any sense of relief is quickly squashed by the thought of the thirsty heating oil tank in the basement that makes even the biggest SUVs seem fuel-efficient.

But someone is making money off the average consumer. The question is: Who?

Retailers, refiners and tax collectors each take a piece of the pie, but lately that share has shrunk to less than one-quarter of the total cost of a gallon of gasoline.

Experts say the rest of the price at the pump is attributable to the cost of crude oil. And it’s the people, companies and countries that “own,” extract and trade in crude oil that are pocketing most of the windfall from the recent surge in fuel prices.

“Petroleum is a worldwide product, so it is part of a worldwide commodity market and really is driven by supply and demand,” said John Kerry, who heads the Governor’s Office on Energy Independence and Security.

So where, exactly, is all of that money you are spending actually going?

It’s not going to the local gas station owner — or at least not much of it anyway. Nationwide, about 11 percent of the cost of a gallon of gasoline went to retailers and their suppliers in July, according to the most recent figures from the U.S. Energy Information Administration.

But that number seems to fluctuate almost as much as the price of crude. Three months ago, the EIA estimated that retailers and distributors accounted for 5 percent of the price of a gallon. In 2004, they recouped, on average, 12 percent of the total cost.

“They are very, very small margins on motor fuels,” said Chris Jackson, vice president of the Maine Oil Dealers Association. “It’s different everywhere, but on average these folks are making pennies on the gallon.”

Instead, gas stations earn most of their money by persuading customers to fuel up with coffee and a doughnut for breakfast, a sandwich, chips and soda for lunch or that over-sized chocolate bar and a six-pack on the way home.

How about taxes?

The exact percentage claimed by the taxman varies by state, but nationwide taxes accounted for about 10 percent of the price of a gallon of gas in both May and July, according to the EIA.

Maine collects 28.4 cents in excise taxes per gallon, plus about 1.4 cents per gallon in environmental fees to help pay the costs of cleaning up groundwater sources contaminated by gasoline or gas additives. The feds take another 18.4 cents a gallon, bringing the total taxes on a gallon of unleaded in Maine to just over 48 cents.

That puts Maine in the upper third nationally. California and Connecticut drivers shell out the most in taxes — 74.9 cents and 70.8 cents a gallon, respectively — while Alaskans pay the least at 26.4 cents, according to the American Petroleum Institute.

State and federal taxes on diesel fuel are slightly higher, meaning truck drivers and others who use diesel in Maine pay 54.7 cents per gallon.

Because gas taxes are collected per gallon pumped — not per dollar spent — the state of Maine still receives about 30 cents a gallon whether the placard outside reads $1.25 or $4.25.

But the rising price of gas has implications for governments — and drivers.

Higher gas prices mean people are driving less and trading in the SUV for a fuel-efficient sedan, resulting in fewer tax revenues flowing into the coffers that finance road and bridge repairs and construction. This comes at a time when the price of asphalt has more than doubled and many Maine roads are in poor shape.

Bruce Van Note, deputy commissioner for operations and budget at the Maine Department of Transportation, said gas tax revenues were down 4 percent for the fiscal year that ended June 30. Budget writers had anticipated last spring that revenues would rebound 2 percent this year.

“A lot has changed since then,” Van Note said. Another 4 percent decline this year would translate into $14 million less than anticipated, he said.

After taxes and retailer-distributors, refineries claim the next chunk. But the percentage of cash flowing to refineries has shrunk significantly in recent years even as the price of crude tripled.

From 2004 to 2007, the refineries that turn the dirty, dark-colored crude into clear gasoline accounted for about 17 percent of the cost of a gallon of gas. But the figure claimed by refineries had shrunk to 10 percent by May 2008 and then to just 3 percent in July, according to the EIA.

The remaining 76 percent of the money that drivers are forking over to fill ’er up is dictated by the price of crude. That means that even if you took away all of the taxes, refining and retailing costs, a gallon of gas in Maine would still cost $2.80 a gallon at today’s prices.

So, again, who is getting rich?

Observers say the biggest beneficiaries of the windfall are the companies — or countries — that either own the crude or pump it out of the ground.

“Big Oil,” the name given to companies such as ExxonMobil or Chevron, has received a lot of the blame as the public watches those companies reap record profits as Americans shell out more and more for gas.

But the American Petroleum Institute points out that these brand name “Big Oil” companies control a tiny percentage of the world’s oil supplies. The vast majority are controlled by government-run companies, such as Saudi Arabian Oil Co., Kuwait Petroleum Co. and Petroleos de Venezuela.

Then there are the traders or “speculators.”

The price of a barrel of crude is set at commodities exchanges such as the New York Mercantile Exchange and on unregulated markets. Traders essentially bet on the fluctuations in the price of oil just by buying futures contracts, hedging that the price will rise and they will profit.

Most times, these speculators — which can include the big investment funds that many Americans hope will allow them to retire comfortably — don’t ever take possession of any oil. They sell the contract before it comes due, hoping that they will pocket more than they paid.

The problem, some experts say, is that traders are artificially inflating the price of oil.

“We are in the throes of some speculation price problems,” said Jamie Py of the Maine Oil Dealers Association.

China, India and other developing nations are consuming more gasoline and diesel, thereby increasing global competition for crude oil. Finally, OPEC, the Organization of the Petroleum Exporting Countries, is actually reducing the amount of crude it pumps.

As if that weren’t enough, there are also a host of other factors that drive up the price of crude, including a weak U.S. dollar, political instability in oil-producing countries, such as Nigeria, and hurricanes that knock out oil rigs in the Gulf of Mexico and refineries along the Gulf Coast.

The price of a gallon of heating oil is in some ways less complicated, and in other ways more so.

Heating oil used by residential customers is not taxed by either the state or federal government. But distributors take a larger share — 24 percent in 2006, the most recent figures available — because of the costs of transporting the fuel to homeowners.

Unlike retail gas stations, most heating oil suppliers can recoup costs by selling other items, Py said.

So why is heating oil consistently more expensive than gasoline?

Residential heating oil is traded the same as diesel on the commodities market, which helps explain why the prices of the two are often similar.

While diesel and No. 2 heating oil are refined less than gasoline, there is a much bigger worldwide demand for diesel than for the gas that most Americans pump into their cars. As a result, the price of diesel — and, therefore, heating oil — is more expensive than conventional gas.

“Because diesel fuel and heating oil are the same product on the commodities market and there is high demand for diesel fuel worldwide, that is making the price of heating oil increase,” Py said.

Kerry with the Maine Office on Energy Independence and Security said consumers must focus on the fundamentals of the marketplace by reducing demand through conservation — whether driving less or using less heating oil.

“And it works, it’s been having a real impact,” Kerry said.