I would start with the guiding principle that read-only decoding is more important than annotation or editing; non-patent encumbered formats should be the first priority (and in Firefox, perhaps the only formats supported); and formats that exist and are in use currently are more important than historical document formats. I don't think this is egregiously bad mission creep for a browser.