Bad zoning?

I'm pretty much a rookie with most art things and so far for my game I'm doing everything (scripting, programming any changes to the engine, all DTS and DIF work). My degree is programming but on my team which is primarily a volunteer group operating under an indie name there's nobody that can handle the art side so I'm working on that.

I say this primarily to give reason to the fact that my dif has light leaks and the texture job is mediocre (I will get around to fix it, just primarily focusing on prototyping a lvl).

My question is caused by the fact that I wanted to see what I could/could not do with the foliage replicator. I threw in several types of plants, and wanted to see what I could do with making grass appear dense (I knew it would suck resources, but with culling and distance modifying I wanted to see what I could come up with). So I added a couple different copies of very low size images of a blade of grass, added some swaying and blending, and wanted to check out the framerate. In order to fully test how much I'd need to tweak it I wanted to know how it affected framerate, so I did some research and found "metrics(fps);" from the console. For now I got this image below. My framerate averages at about 50 and my computer's a pretty good powerhouse so it's pretty obvious I'm going to have to make some changes. However, one thing I noticed is that if I swing my camera towards this DIF, my framerate plunges to between 15 and 25.

This dif has 57 portals and 9 zones. At first I thought it seemed to have something to do with peering through these 2 portals. If I actually go into the throne-room, it jumped to ~60. Walking around the castle it's usually 40-60. Occasionally, however, it drops to ~20 at different parts of the castle and facing different directions. I'm confused as to what's causing it. Most of the rooms in my Dif have nothing in them yet. All the rooms however that are populated with a bunch of DTS objects or have a large poly count within the room (like stairs) are just fine. Granted I still don't know a lot of modeling and texturing things like bumpmapping, normal maps, diffuse, heightmaps (if anybody knows a good place that explains these with tutorials I would appreciate it), but as far as I know I've done most things correct on my DIF. Aside from the fact that I could also use a good DIF practices tutorial, I need to understand what's causing this.

Any help is greatly appreciated!

EDIT: Also tested disabling all the foliage I had (it was ALOT). All directions but facing the castle jumped by 30fps, however, looking at the castle jumped 5fps. I could look around at the inside of the walls of the castle perimeter and it would be between 90-100fps. The second even the corner of the castle dif came into the screen, it dropped to 60. The second the front door to the castle came into view, it bottomed at about 20fps.