Bamboo shrimp have no interest in hiding in caves, unless there happens to be a strong current in it. They need a current to filter and obtain their food(otherwise you'll just see them wandering around your tank, picking up food off the substrate. You said he is laying down? On his side? If so, not good news I'm afraid and won't last much longer. Have you tested your water? Shrimp cannot abide by ANY ammonia or nitrites and it will kill them off quickly. Since Bamboo shrimp mainly eat small particulate matter, they prefer an established tank. That's not to say you can't put them in a newer set up but there cannot be any ammonia present whatsoever. I put mine in my tank while it was new but I also feed them powdered spriulina and swished it around in the water. Plus I do weekly water changes of 50%. You should just leave him be. Test your water and do a 50% water change if you haven't done one for the week. If there is ammonia present, you need to do 70% then later on, do another water change. If you don't have a test kit, go get one. If you don't want to get one, start doing some water changes because I can assure you that there is something wrong with the water. Also, what is the temperature at? They don't like anything higher than 78F. I have my tank at 76F because I have a community tank but personally would put it lower for the shrimp if I could.
Good luck