PNG support was introduced to .ico format alongside Vista in ~2006, you can certainly use it as a multiple-png-only format.

Wikipedia has a description of the file format and it's certainly not the worst i've seen - restricting yourself to only PNG images removes all the mess around skipping bmp headers and paletted colours, you end up with a very short and concise header and binary list format, and it should be trivially implemented from the description.

Of course, i havn't implemented it myself, so it's obvious i'd say that. But i'm curious as to what issues you have with it / what you would change?