They can eat Grape Vine leaves, dandelion leaves and such. This is fine for keeping them alive for a week or two. But I found that it was best to feed them at least 50% mulberry based food if you want to keep them alive for a full life-cyle / breed. Also, if they've only had mulberry for the first weeks of life, you may find they resist switching to alternate foods right away.
Every time I have them I give them Dandelion leaves to make the mulberry paste last longer. They seem happy to have proper leaves to me and love it every time I offer it
Anthropomorphism towards insects aside I think there are some of them that just stop eating when the paste runs out too.