I've heard this plenty of times, but is it actually true? I found it surprisingly hard to get real fuel consumption figures at various speeds but two sets of data suggest that it is total false, and that a truer statement would be 'My big car uses loads more fuel that a small car going through town and continues to do so going down the motorway.'
The website of the German motor magazine 'Auto Bild' has an interesting comparison of the fuel consumption of various cars recorded from 80 km/hr up to the car's top speed. I've summarized the data for the petrol driven vehicles below.
Sticking to speeds legal in Switzerland (i.e. up to 120 km/hr) you can see that the relative order of the consumption remains pretty much unchanged between 80 and 120 km/hr. An interesting exception to this are the Mercedes C180K and SLK200K which really do become relatively more economic, assuming that you drive fast enough. If I find out why these cars have such flat curves I'll let you know.
Anyway, you can find the original article at http://www.autobild.de/artikel/der-auto-bild-verbrauchs-test_55631.html and a more detailed analysis of it at http://www.greencarcongress.com/2006/05/fuel_consumptio.html
Another set of data from the 'Center for Transportation Analysis , Oak Ridge National Laboratory' and summarized on Wikipedia http://en.wikipedia.org/wiki/Fuel_economy_in_automobiles#cite_note-ornl-10 shows a similar picture concerning relative economy, and covers lower speeds as well.