Game Development Community

cartoon outlining (cartoon rendering part1)

by F.W. Hardijzer · in Torque Game Engine · 05/31/2003 (3:22 pm) · 113 replies

Ok :)
I've been working on this for about a whole day, and still there are some bugs in it :(
But that doesn't hold me from posting it for others to use :)
so here is the code:
FOR SHAPES
in ts/tsMesh.cc
in TSMesh::render after:
S32 drawType = getDrawType(draw.matIndex>>30);

      glDrawElements(drawType,draw.numElements,GL_UNSIGNED_SHORT,&indices[draw.start]);
   }
add
/*cell shading*/
   bool doOutline=Con::getBoolVariable("$Pref::renderOutline",true);
   
   if (doOutline) {
      bool oldlighting=glIsEnabled(GL_LIGHTING);
      glDisable(GL_LIGHTING);
      F32 oldLineWidth;
      int oldcullface;
      glGetIntegerv(GL_CULL_FACE,&oldcullface);
      glGetFloatv(GL_LINE_WIDTH,&oldLineWidth);
      F32 lineWidth=Con::getFloatVariable("$Pref::OutlineWidth",2);
      glLineWidth(lineWidth);
      glCullFace(GL_FRONT);
      glPolygonMode (GL_BACK, GL_LINE);
      glDepthFunc(GL_LEQUAL);
      bool old2dtex=glIsEnabled(GL_TEXTURE_2D);
      glDisable(GL_TEXTURE_2D);
      F32 oldcolor[4];
      glGetFloatv(GL_CURRENT_COLOR,oldcolor);
      glColor4f(0.0f,0.0f,0.0f,1.0f);
      for (S32 i=0; i<primitives.size(); i++)
      {
         TSDrawPrimitive & draw = primitives[i];
         AssertFatal(draw.matIndex & TSDrawPrimitive::Indexed,
                  "TSMesh::render: rendering of non-indexed meshes no longer supported");

         S32 drawType = getDrawType(draw.matIndex>>30);
         glDrawElements(drawType,draw.numElements,GL_UNSIGNED_SHORT,&indices[draw.start]);
	  }
      glColor4fv(oldcolor);
      if (old2dtex) glEnable(GL_TEXTURE_2D);
      glDepthFunc(GL_LEQUAL);
      glLineWidth(oldLineWidth);
      glColor3f(1.0f,1.0f,1.0f);
      glCullFace(GL_BACK);
      glPolygonMode (GL_BACK, GL_FILL);
      if (oldlighting) glEnable(GL_LIGHTING);
   }
   /* end cell shading*/

FOR TERRAIN:
in terrain/terrRender.cc
in TerrainRender::renderXFCache after
U32 vertexCount = mXFIndexBuffer[count + 1];
      glDrawElements(mode, vertexCount, GL_UNSIGNED_SHORT, mXFIndexBuffer + count + 2);
add:
//Cell shading
   /*cell shading*/
   bool doOutline=Con::getBoolVariable("$Pref::renderOutline",true);
   if (doOutline) {
		/*extra cell shading*/
	   F32 oldLineWidth;
	   int oldcullface;
	   glGetIntegerv(GL_CULL_FACE,&oldcullface);
	   glGetFloatv(GL_LINE_WIDTH,&oldLineWidth);
	   F32 lineWidth=Con::getFloatVariable("$Pref::OutlineWidth",2);
	   glLineWidth(lineWidth);
	   //glBindTexture(GL_TEXTURE_2D,NULL);
	   glColor3f(0.0f,0.0f,0.0f);
	   glCullFace(GL_FRONT);
	   glPolygonMode (GL_BACK, GL_LINE);
			//Draw here
	   glColor3f(0.0f,0.0f,0.0f);
	   glDisable(GL_TEXTURE_2D);
	   glDrawElements(mode, vertexCount, GL_UNSIGNED_SHORT, mXFIndexBuffer + count + 2);
	   glEnable(GL_TEXTURE_2D);
	   glLineWidth(oldLineWidth);
	   glCullFace(GL_BACK);
	   glPolygonMode (GL_BACK, GL_FILL);
		/*end extra cs*/
   }
   /* end cell shading*/

FOR INTERIORS:
I'm still working on this one, and it's quite difficult (since there are lots of render codes in the interiors,
I've finally gotten some lines over it, but it is pretty glitchy, and will only look good on some interiors,
the $pref::renderOutline for interiors is $Pref::Interior::renderOutline,
and $pref::outlineWidth is $Pref::Interior::outlineWidth...
the reason i changed these vals for interiors is because i can imagine someone wanting terrain and shape, btu not this buggy interior code enabled :)
anyways, enough bullshit, code:
in interior/interiorRender.cc
in void Interior::render
after:
if (smRenderMode != 0) {
      PROFILE_START(IRO_DebugRender);
      debugRender(pMaterials, instanceHandle);
      PROFILE_END();
      return;
   }
add:
//cartoon outline
   bool doOutline=Con::getBoolVariable("$Pref::Interior::renderOutline",false);
   
   if (doOutline) {
	   F32 oldLineWidth=1;
	   glGetFloatv(GL_LINE_WIDTH,&oldLineWidth);
       F32 lineWidth=Con::getFloatVariable("$Pref::Interior::OutlineWidth",2);
       glLineWidth(lineWidth);
	   // Base textures
	   glBlendFunc(GL_ONE, GL_ZERO);
	   glDisable(GL_TEXTURE_2D);
	   glColor3f(0, 0, 0);
	   for (U32 i = 0; i < sgActivePolyListSize; i++) {
		  const Surface& rSurface = mSurfaces[sgActivePolyList[i]];
		   glBegin(GL_LINE_LOOP);
		   glVertex3fv(mPoints[mWindings[rSurface.windingStart]].point);

		   S32 skip = rSurface.windingStart + 1;
		   while (skip < (rSurface.windingStart + rSurface.windingCount)) {
			  glVertex3fv(mPoints[mWindings[skip]].point);
			  skip += 2;
		   }

		   skip -= 1;
		   while (skip > rSurface.windingStart) {
			  if (skip < (rSurface.windingStart + rSurface.windingCount))
				 glVertex3fv(mPoints[mWindings[skip]].point);
			  skip -= 2;
		   }
		   glEnd();
	   }
	   glEnable(GL_TEXTURE_2D);
	  //return;
   }
   //End cartoon outline

some prefs of this:
$Pref::renderOutline (true/false) render outlines or not
$Pref::outlineWidth (0-10) linewidth of outlines
You are free to use this code for whatever project you'd like, as long as you post every change you make on this forum so others can use it too :)

BUGS:
- enviroment mapped models have 'reflective' lines

Please if you like this stuff, contribute and help us with this!
also if you know anything about cell-shading, try to do that too...
in other words:
FRANK BIGNONE: please post your cell shading stuff from DOP here :(

and another word, noone is allowed to submit this as a resource except for me when i think the code is good enough to be posted as a resource :)
Page«First 1 2 3 4 5 6 Next»
#101
04/20/2005 (6:42 am)
Chris,
As you can see this is not model dependant. What I am finding in my research is that using the Push modifier in max to create an outline and use that for the model is faster and easier than the current code. As a solution might be to use the push modifier for the main models and the code for elements such as terrain and low-rez models/important ones.

Also, mind making the images smaller, so we have less loading time on this thread, it's allready getting pretty big.
Thanks!
Toby.
#102
04/20/2005 (6:46 am)
I would agree. However, that is not a viable solution to anyone who is running large numbers of polys already, since that technique will effectively double your poly count... It does look damn good though... I guess what it comes down to is the terrain and interiors. Since your gam edoesn't appear to care about interiors Toby, I would suggest leaving the interior code out (unless it is bug free) as well as teh terrrain and just NOT outlining the terrain. Ninjasticks doesn't appear to need a whole lot in the terrain department, so the lack of outline for it probably isnt going to matter. Your ninjas will look pretty cool though.
#103
04/20/2005 (6:54 am)
Chris,
Ninjasticks does require the terrain code and interior code, as some of our missions are inside and some totally out. I refained from using the interior code on the screenshots as this is still very buggy. The main cause of the problem is due to performance. When I display a couple of models with cel-shading on, it just kills the FPS. Which is why a push-created outline for the models is less of a perfomance hit and have more control (as I can set the color and give it soem artistic tweaks). While the interior and Exterior still would make use of the cel code as this would be, as far as I know, the only way.

Toby.
#104
04/20/2005 (7:07 am)
The push method will kill performance once you get enough models in the scene. Just be careful... You are doubling your poly count for everything you do this to.

Are you doing all of the cel code for the game or is it someone elses job? The reason I ask is if you don't have someone working on this I would be interested in taking on the challenge, at least in part. My time is quite restricted, but I think it might be worth my time.

I would seriously suggest using a different approach for the terrain outlines. As for the interiors... If you are careful and draw your lines in obvious places, like corners, you can probably fake it fairly accurately. Mind you it will take a talented texturer to line up the lines properly. This is definately tha approach I would take.

For terrain, the first thing that comes to mind is since the terrain tiles are easiy accessible (they are procedurally placed and you can access them in order) I would procedurally go through each pair of connected tiles, that are in a viewing range, and do a dot product on the players position to the normal of the terrain tile. If the dotproduct of one is + and hte other is - then you need to draw that edge. That will work, just make sure you do it with a viewing range to limit how many tiles you are doing each pass..
#105
04/20/2005 (7:17 am)
Ill reduce the size when I get home, also I'm going to look in the stencil shadow resource for the edge detection code and see if I can find a solution from that. Still the best method for outlining is the use of shaders, but that's not an option with TGE.

*Edit
Even better, i just changed the format to jpg.
#106
04/20/2005 (7:38 pm)
There are 2 other solutions im working on.
1. Using the stencil buffer, google for it and you will find lots of info about using it for outlines. (im having probs with it)
2. Using the edge detection code from the stencil shadow resource and draw lines on the edges.
#107
04/21/2005 (3:35 am)
Well using the stencil buffer is probably not a cross platform way of doing things. If I remember correctly it is a D3D way of doing things... could be wrong though
#108
04/21/2005 (4:16 am)
Marble blast uses it for shadows, the mirrors in TGE uses it, and best of all John Carmack uses it for his OpenGL engine.
#109
08/14/2005 (2:09 pm)
@F.W. Hardijzer
I know I'm reviving an old thread here, but I've tried out your code (which is incredibly awesome, by the way =P) and I've run into one problem (probably a simple one).

With the exception of the orcs, everything in the Torque Demo has black lines. The orcs have white and blue lines when they are close to the camera, though. I have tried messing around with some of the colour matrices in the code, but I haven't been successful.

Is there any way I could fix those lines? I am somewhat new to OpenGL...
#110
08/15/2005 (4:02 pm)
Has anyone had this problem with the shape cel-shading code?
#111
09/02/2005 (12:25 pm)
Danny, I've that problem too. I think It's the environmental mapping! ;) I've just begun trying this fake cel shading solution, so I don't know how to fix it. I'm using an ATI, just in case...
#112
09/03/2005 (9:57 am)
I've actually found out my problem (nobody really responded, so I didn't reply back).

I've just changed all of the 1.0s in the glColor3fs and glColor4fs to 0.0s. It didn't work the first time I tried it, but it did now. That was how I fixed it.
#113
12/18/2005 (3:43 am)
So what's the code fix for this? Does this relate to the Environment mapping or bugs in the video drivers?

Toby.
Page«First 1 2 3 4 5 6 Next»