Menus > Layers menu > 
Layers > Delete > Unused layers
 
 
 

Deletes all layers with no objects on them.