But we were not talking about the 1970s! All the latest diesel engines are far more efficient than those of only a few years ago.
Have you noticed how big & heavy cars are getting?
Ka's are the size of the original Fiesta, Fiestas are as big as Escorts, Focus's are the size of the original Mondeo, and Mondeos are bigger than Scorpios.
And that's just been over the last 20 years. All that metal, plus all the electronic motors for seats/windows/sunroof/central locking and mirrors, air-con systems, and huge wheels. Not to mention airbags & side impact beams...
A lot of the modern cars get good results on the test because of stop/start, which isn't an improvement in 'engine efficiency' as such (anyone can switch off the engine when idling), but they are now using electric power steering, and regenerative braking, and alternators which only draw current as needs be. Also, many autoboxes (such as the dual-clutch type) do without torque convertors, which reduce efficiency.
Its all to get a low VED & company car tax rating.
I'm not saying a powerful engine can't be driven economically (I have a 3.2 & regularly see over 40 mpg), but I do have to over-ride the autobox & shift up early, etc.
In the 'real-world' (i.e. not govt test conditions), the biggest gains in fuel efficiency have been made, at the lowest cost. Manufacturers will now need to be particularly inventive in using non-traditional materials to reduce kerbweight (which will drive up the cost of the cars), to improve their products.