
So, I'll use a chemical to melt the doors and windows into the surface.
Markers are great for marking Styrofoam. In the picture to the left, I have marked out the various levels in black and the stairs in red. I used dry erase markers after testing them to make sure they wouldn't damage the material. I also outlined the windows and doors before starting with my Sharpie.

One caveat here. Not all markers are the same. In the last image series called Rough Cuts, I used a fine line red Sharpie to mark my cut outs. It didn't melt anything. A fine line marker maybe too small, formulated differently or the age of the marker cause it not to melt the Styrofoam. I picked an old marker for this task because this chemical does a number on the nib of the marker.
I cannot suggest alcohol based markers for this. I did try it and it didn't work. First, you'll ruin a perfectly good marker if it does work, and to my knowledge, alcohol doesn't damage Styrofoam. It is often used as a thinner in airbrushes, and airbrushes work fine on Styrofoam models. It's the ink doing the burning, not the smelly carrier.

The marker is easily controlled and my lines are sort of straight. My purpose in making an indented windows and doors is to protect and mark those areas from the plaster I will skim over the surface. Later today, I will make a series of light cardboard masks for these spaces, with pull tabs so I can remove any mistakes I make with the plaster.
These windows and doors will have wooden or plastic features and I will need to be able to partially recess them in these spaces.