Everyone knows how books work. You have a great idea, write a synopsis and a couple of chapters (or all of it), send it off to a publisher (or twenty), sign a contract, get the first chunk of the massive (or tiny) advance, sit down to write the book, get the next chunk of advance, argue about a few changes, get the last chunk of advance, and watch your book walk (or crawl) off the shelves while you wait for your first royalty cheque (after it earns out, of course).
Sit at home looking at your email until a publisher asks you to write a book. Send in a synopsis, get first chunk of small (or large - not unheard of) fee, write book, get second chunk of fee, argue about changes to keep it in line with rest of series, get the last chunk of fee and expunge book from your memory totally while waiting for the next email from a publisher. It's more like a series of one-night-stands than a proper relationship; writing whore rather than the kept mistress of RandomPenguin (or wherever).
Lots of books are written for a flat fee, especially in children's publishing. Typically, you might be offered a flat fee for children's non-fiction (any market from textbook to mass market); reading scheme books for the schools market; licensed character books (those about established characters such as Wallace and Gromit or Disney mice); and character-led fiction that's part of a series written by many authors and published under a pseudonym that reflects the genre (eg Carnivorous d'Vil or Tinsel Rosebottom). You could can get a royalty deal on all these from some publishers - but a flat fee is fairly typical.
There are obvious advantages for publishers in paying a flat fee: they know how much they have to pay; there is none of that complicated looking at the spreadsheet to see how much to pay you in royalties; if your book is a mega-hit, they keep all the profit; it makes their budgeting easy.
There are less obvious disadvantages for publishers: it makes no material difference to you whether the book sells, so you won't do much publicity; they certainly can't expect you to do any visits or signings or anything for free as increasing the sales of the book does you no good (except for BookScan figures, but they don't really count for much in these particular markets). If a book is really successful, the writer won't want to write for you again on a flat fee basis as they will suppose (or imagine) they can earn more writing for another publisher who will pay a royalty.
There are obvious disadvantages for the writer: if your book sells 100,000+ copies you still don't get any more than the flat fee (been there). You won't know what the sales figures are - this might sound unimportant, but it's not. If you want to pitch an idea to another publisher, it can be very useful to be able to say a previous title sold 100,000 copies. Of course, the publisher can check that figure, but you can't! (Unless you pay Neilsen for the info.) And because you can't, you won't know which book to list as your mega-seller.
But there are also less obvious advantages for the writer: you don't need to keep checking sales rank on Amazon, look for reviews or worry about whether the book is in the shops; you don't have to do visits and signings (unless you like that sort of thing); your mind is free to work on the next book; you can do your budgeting with confidence; the flat fee is likely to be more than the advance you would have got for the same book. A book with a small advance that doesn't earn out will generally pay you less than the same book for a flat fee. If you want money now to pay to feed your children and pay the broadband bill, flat fee is often the more attractive option. It's not selling out, it's pragmatism.
Often, but not always, a flat fee is offered when the book is the publisher's idea. There is a certain fairness to this - the book would be written whether you wrote it or not. Maybe not as well, but frankly it's not going to sell on quality, is it? If a kid wants the next Vampire ballerinas book, they'll buy it anyway. As long as the flat fee is fixed with a realistic and fair assessment of the likely sales, it is usually a reasonable way to operate. The author should get as much as they would get in total royalties if it were a royalty deal.
Of course, there are rogue publishers who try to get away with flat fee offers of £200 or some such. Good, professional writers turn them down, but probably somewhere an aspiring writer will take the deal. And it takes guts to turn down anything when you don't have the next mortgage payment in the bank. (If the publisher is small and can't afford a decent fee, that's no excuse for offering £200 - they should at least make the £200 an advance and give a decent royalty; then it's up to the writer whether they trust their work and the publishers' distribution enough to give it a go.)
Flat fee is often 'work for hire' in legal terms. The publisher frequently wants to buy out the copyright and will often try to get you to waive your moral rights as well. The general advice is never to sell copyright, only to licence it. Whether you will get that through depends on the type of book and the publisher. (And how much you are prepared to dig your heels in.)
Imagine a publisher has a series of books about vegetables. They have asked you to write the Potatoes title. You want to keep the copyright. They will find someone else. Any competent writer can write about potatoes. Maybe some won't do it quite as well as you will, but the book will still be done and the editor can make it good enough. The publisher doesn't need to give way on the copyright issue.
Now imagine a publisher has a series of early reader fiction for sale into schools. They have asked you to write some animal stories. The plot and characters are up to you, they just have to be about animals, be 500 words long and divide into 12 spreads. You want to keep the copyright. This is a harder thing to do well, so the publishers might well agree. There is no end of people who will claim they can write animal stories 500 words long, but most of those stories will be rubbish. It's not like potatoes. (By the way, I have done both of these, and so feel quite free to diss the potato-writing aspect. Although actually I didn't do the potato title, so maybe that one was harder than carrots.)
If you can keep copyright, do so. For one thing, if the publisher goes bust you should be able to claim your title back and publish it elsewhere [hollow laughter] or make an ebook [unless it's illustrated, not by you]. I always cross out moral rights waiver if my name will be on the book (if your name's not going to be on it anyway - as in licensed character work -
the moral rights are of no value to you).
As for the one real injustice - the 100,000+ sales because you did the book so well - my solution to that one was to negotiate with the publisher to add a royalty clause to all future contracts that kicks in if sales go over 100,000. They are content with that - if the book sells that many copies, they can afford the royalty, and if it doesn't they've not lost anything.