Do curling leaves mean underwatering?

Often, but not always. Curling can also come from heat, low humidity, pests, or root stress.

Will curled leaves uncurl?

Mild curling may improve. Damaged or hardened leaves may stay curled, but new growth should look better.