Add or remove objects during animation?

(I made it an answer because comments don't allow for enough characters, and I added my first comment for good measure.)

You can animate the Show In Renders located in Object Properties > Visibility and keyframe the checkbox. Or animate and keyframe the Disable in Renders (little camera icon) in the Outliner. If the camera icon is not there, you can activate it in the Outliner Filter (little funnel icon). Note: Show In Renders and Disable in Renders are the same setting in different locations

The only other thing that comes to mind at the moment is giving the objects materials that can be animated to be invisible. It would be easiest if all objects had the same material. Though you could link the invisibility behavior of different materials with drivers. Here are two answers on how to do something very similar. However, I'm not sure (doubt even) if it will increase FPS and/or decrease render-time.

Also here is a small discussion with some other workaround methods.

Edit:

Just thought of another idea. Most probably it won't work for your situation, but i thought I'd mention it anyway: You could also maybe use instancing. All children objects would need to be the same, and can't be individual objects.

You could parent a single child object to an object consisting of loose vertices instead. The go to Properties > Object Properties > Instancing, and click Verts. You would see a mesh of the child object for every vertex in the parent object. Then you can animate the Disable in Render of a single child object to make them all disappear (or do any other localized animation for that matter).