The matter doesn't matter

Genesis just rearranges matter into the stuff of life. The planet they called Genesis was actually composed only of the hulk of the Reliant. However, there are at least two problems with this: One is the size of the planet - it should have been exactly the same mass as the ship. Another is that there seemed to be a sun nearby, whereas there wasn't a sun near the Reliant.

Matter Transference

Right, mass is the big question mark if the Genesis transformed the Reliant into Planet Genesis. But that still leaves the planet in the middle of the Mutara Nebula. If the explosion was great enough to affect the target planet, then maybe it was great enough to also create a star so that life may thrive. That brings us to a whole new problem. Now we have to find a planet that is isolated from everything, or one in an area space consisting of only dead planets. Cant be rearranging life on a nearby populated planet, can we? Or is that a whole new pick in it's own right?

It was the Nebula

I don't know how they managed this, as nothing is said in the film about the Genesis device being able to change the nature of matter. However, after the device detonates the Mutara Nebula no longer exists, so I have to assume that the Nebula and everything in it (including the Reliant) became the Genesis planet.

Genesis planet created from..

The Genesis planet was created from the matter in the Mutara Nebula, that's why, just before the explosion it's all colorful in space, but right afterward, it's black space with stuff moving toward the explosion- it's sucking in matter

re: comments

but then why are they even looking for a planet at the beggining of the movie? I'd use this as a great opportunity... find a nice spot in space, bring the garbage of the entire quadrent out to it... fire off a genesis torpedo, and BAM... garbage heap = new planet

Regula 1 is Genesis...

Both the Reliant and the Enterprise had thier WARP drives damaged and could not move very far or fast. The blast radious of Genesis encircled Regula 1 which was a "rock in space" thereby creating the Genisis Planet. The Genesis Cave was underground and not exposed to the matter transformation process.

Comment 1651 is correct

The name 'Genesis Planet' is not used in reference to the planetoid that Regula One orbits. It's name is Regula. 'The Genesis Planet' is the name of the planet that was formed from the matter comprising the Mutara Nebula. When the Genesis device detonates inside the nebula, you can see in the film that the nebula disappears. Its matter is sucked into the explosion and immediately formed into a planet.

Genesis not Flawed

I understand that Genesis used all matter in it's wake to create the Genesis planet. One thing not noted is Dr.Carol Marcus,(Kirks ex)said specifically there cannot be as much as a microbe present. The test site has to be completely lifeless. So how does Genesis become a flawed experiment according to Savvik in ST:III? The Reliant was full of microbes from all the bodies lying around. So shouldn't the test of the Genesis Project be used on a lifeless planet or rock. The fact the project used a nebula and the wreckage of a starship to create, flawed the experiment and David may in fact have used protomatter correctly.

Comment on comment 51567

Dr. Marcus said there couldn't be "so much as a microbe" simply because the Genesis effect would destroy any life already existing where it took place, as Spock put it, "in favor of its new matrix." As far as the protomatter is concerned, we just have to accept that David misused it, because he said so. The evidence of Genesis' flaw(s) was the instability of the planet itself (surely you noticed that).

Wrong!

The Genesis planet and it's star were formed from the matter in the nebula, not the remains of the Reliant. The Reliant was obliterated in the blast. We now have a theory that in the formation of the universe, stars and planets were made by the existing protons and neutrons, etc. which first had to form atoms, and then gasses and then the complex molecular arrangements that make up the material of planets. Stars were also formed from the same atoms in different reactions. Therefore, the Genesis device does this without needing to form the original atoms, since they already exist in the nebula, the planet and a star to keep it alive are formed!\rThe planet could not exist without a star to keep it's functions alive so the reaction adjusted to compensate for this. Now, why the star didn't have problems like the planet was going to have is a nitpick since they were both flawed from the adjustment Kirk's son had to make.