When a vApp is powered on, we look at vCenter and see if Networks (port groups) by name already exist. If they do, then we try to delete them to ensure we start from a clean slate.
If we get to a clean slate, we will provision a new Network (Port Group) to be associated to the vApp. If the deletion fails, typically when in use, then the power on operation on the vApp fails.
What it really sounds like is that the vApp is getting powered off and the Edge (and therefore Network / Port Group) is being Stranded.
Brand New vApps would not see this issue, because the Networks would never have existed.
Proper clean up would be from Stranded Items to ensure the Edge & Network are removed from vCloud and from vCNS/NSX. If you just delete it from vCenter, there could be left over items (stranded in vCD) and networks in the vCNS/NSX database (VXLAN only).