improve broken shell detection, and give a better error message