Howdy amigo, I bought a 2001 Espace 2 weeks ago and I can report that, yes. they are unreliable.
Don't let that put you off though, it is the electrics that are a big pain in the bum.
I can't comment on the mechanical side of a 2004 Espace as mine is 2001 and built by an entirely different factory (Matra - not Renault).
I can say that the older one certainly feels solidly built and doesn't seem any worse than any other car for mechanical reliability.
Just go over it with a fine tooth comb, check absolutely everything electrical, no matter how long it takes.
If there are any problems it's probably wise to walk away as getting them fixed can be ridiculously expensive.
Hopefully Espace IV owners on the forum will pipe up and give you some common faults to check on newer Espaces, even a non owner may be able to give a clearer view as I can't speak for the newer ones.
Please don't be put off by my comments as they are good to drive the kids about and are very safe.
Just make sure you pick it apart in fine detail before buying it.
I don't regret buying the Espace but it is a bit annoying, if it seems like the best one for you I'd imagine you've gone through the list in much the same way I have (ie; nah - too small, nah-too small, nah - too ugly, nah - looks like it's going to fall over, nah - it's got no Ncap rating at all, nah - no ABS.....etc etc etc)
There's too many good points to a new Espace over it's rivals to ignore it because it has a reputation.
My opinion is based on what I've read on this forum, what I've experienced with my older Espace since buying it and the competition - which I studied endlessly. Everything was too expensive or didn't measure up space/spec.-wise.
If it works, I'd say buy it, regardless of what I've said here.
Sorry i can't be any more specific.