cartoon outlining (cartoon rendering part1)
by F.W. Hardijzer · in Torque Game Engine · 05/31/2003 (3:22 pm) · 113 replies
Ok :)
I've been working on this for about a whole day, and still there are some bugs in it :(
But that doesn't hold me from posting it for others to use :)
so here is the code:
FOR SHAPES
in ts/tsMesh.cc
in TSMesh::render after:
FOR TERRAIN:
in terrain/terrRender.cc
in TerrainRender::renderXFCache after
FOR INTERIORS:
I'm still working on this one, and it's quite difficult (since there are lots of render codes in the interiors,
I've finally gotten some lines over it, but it is pretty glitchy, and will only look good on some interiors,
the $pref::renderOutline for interiors is $Pref::Interior::renderOutline,
and $pref::outlineWidth is $Pref::Interior::outlineWidth...
the reason i changed these vals for interiors is because i can imagine someone wanting terrain and shape, btu not this buggy interior code enabled :)
anyways, enough bullshit, code:
in interior/interiorRender.cc
in void Interior::render
after:
some prefs of this:
$Pref::renderOutline (true/false) render outlines or not
$Pref::outlineWidth (0-10) linewidth of outlines
You are free to use this code for whatever project you'd like, as long as you post every change you make on this forum so others can use it too :)
BUGS:
- enviroment mapped models have 'reflective' lines
Please if you like this stuff, contribute and help us with this!
also if you know anything about cell-shading, try to do that too...
in other words:
FRANK BIGNONE: please post your cell shading stuff from DOP here :(
and another word, noone is allowed to submit this as a resource except for me when i think the code is good enough to be posted as a resource :)
I've been working on this for about a whole day, and still there are some bugs in it :(
But that doesn't hold me from posting it for others to use :)
so here is the code:
FOR SHAPES
in ts/tsMesh.cc
in TSMesh::render after:
S32 drawType = getDrawType(draw.matIndex>>30);
glDrawElements(drawType,draw.numElements,GL_UNSIGNED_SHORT,&indices[draw.start]);
}add/*cell shading*/
bool doOutline=Con::getBoolVariable("$Pref::renderOutline",true);
if (doOutline) {
bool oldlighting=glIsEnabled(GL_LIGHTING);
glDisable(GL_LIGHTING);
F32 oldLineWidth;
int oldcullface;
glGetIntegerv(GL_CULL_FACE,&oldcullface);
glGetFloatv(GL_LINE_WIDTH,&oldLineWidth);
F32 lineWidth=Con::getFloatVariable("$Pref::OutlineWidth",2);
glLineWidth(lineWidth);
glCullFace(GL_FRONT);
glPolygonMode (GL_BACK, GL_LINE);
glDepthFunc(GL_LEQUAL);
bool old2dtex=glIsEnabled(GL_TEXTURE_2D);
glDisable(GL_TEXTURE_2D);
F32 oldcolor[4];
glGetFloatv(GL_CURRENT_COLOR,oldcolor);
glColor4f(0.0f,0.0f,0.0f,1.0f);
for (S32 i=0; i<primitives.size(); i++)
{
TSDrawPrimitive & draw = primitives[i];
AssertFatal(draw.matIndex & TSDrawPrimitive::Indexed,
"TSMesh::render: rendering of non-indexed meshes no longer supported");
S32 drawType = getDrawType(draw.matIndex>>30);
glDrawElements(drawType,draw.numElements,GL_UNSIGNED_SHORT,&indices[draw.start]);
}
glColor4fv(oldcolor);
if (old2dtex) glEnable(GL_TEXTURE_2D);
glDepthFunc(GL_LEQUAL);
glLineWidth(oldLineWidth);
glColor3f(1.0f,1.0f,1.0f);
glCullFace(GL_BACK);
glPolygonMode (GL_BACK, GL_FILL);
if (oldlighting) glEnable(GL_LIGHTING);
}
/* end cell shading*/FOR TERRAIN:
in terrain/terrRender.cc
in TerrainRender::renderXFCache after
U32 vertexCount = mXFIndexBuffer[count + 1];
glDrawElements(mode, vertexCount, GL_UNSIGNED_SHORT, mXFIndexBuffer + count + 2);add://Cell shading
/*cell shading*/
bool doOutline=Con::getBoolVariable("$Pref::renderOutline",true);
if (doOutline) {
/*extra cell shading*/
F32 oldLineWidth;
int oldcullface;
glGetIntegerv(GL_CULL_FACE,&oldcullface);
glGetFloatv(GL_LINE_WIDTH,&oldLineWidth);
F32 lineWidth=Con::getFloatVariable("$Pref::OutlineWidth",2);
glLineWidth(lineWidth);
//glBindTexture(GL_TEXTURE_2D,NULL);
glColor3f(0.0f,0.0f,0.0f);
glCullFace(GL_FRONT);
glPolygonMode (GL_BACK, GL_LINE);
//Draw here
glColor3f(0.0f,0.0f,0.0f);
glDisable(GL_TEXTURE_2D);
glDrawElements(mode, vertexCount, GL_UNSIGNED_SHORT, mXFIndexBuffer + count + 2);
glEnable(GL_TEXTURE_2D);
glLineWidth(oldLineWidth);
glCullFace(GL_BACK);
glPolygonMode (GL_BACK, GL_FILL);
/*end extra cs*/
}
/* end cell shading*/FOR INTERIORS:
I'm still working on this one, and it's quite difficult (since there are lots of render codes in the interiors,
I've finally gotten some lines over it, but it is pretty glitchy, and will only look good on some interiors,
the $pref::renderOutline for interiors is $Pref::Interior::renderOutline,
and $pref::outlineWidth is $Pref::Interior::outlineWidth...
the reason i changed these vals for interiors is because i can imagine someone wanting terrain and shape, btu not this buggy interior code enabled :)
anyways, enough bullshit, code:
in interior/interiorRender.cc
in void Interior::render
after:
if (smRenderMode != 0) {
PROFILE_START(IRO_DebugRender);
debugRender(pMaterials, instanceHandle);
PROFILE_END();
return;
}add://cartoon outline
bool doOutline=Con::getBoolVariable("$Pref::Interior::renderOutline",false);
if (doOutline) {
F32 oldLineWidth=1;
glGetFloatv(GL_LINE_WIDTH,&oldLineWidth);
F32 lineWidth=Con::getFloatVariable("$Pref::Interior::OutlineWidth",2);
glLineWidth(lineWidth);
// Base textures
glBlendFunc(GL_ONE, GL_ZERO);
glDisable(GL_TEXTURE_2D);
glColor3f(0, 0, 0);
for (U32 i = 0; i < sgActivePolyListSize; i++) {
const Surface& rSurface = mSurfaces[sgActivePolyList[i]];
glBegin(GL_LINE_LOOP);
glVertex3fv(mPoints[mWindings[rSurface.windingStart]].point);
S32 skip = rSurface.windingStart + 1;
while (skip < (rSurface.windingStart + rSurface.windingCount)) {
glVertex3fv(mPoints[mWindings[skip]].point);
skip += 2;
}
skip -= 1;
while (skip > rSurface.windingStart) {
if (skip < (rSurface.windingStart + rSurface.windingCount))
glVertex3fv(mPoints[mWindings[skip]].point);
skip -= 2;
}
glEnd();
}
glEnable(GL_TEXTURE_2D);
//return;
}
//End cartoon outlinesome prefs of this:
$Pref::renderOutline (true/false) render outlines or not
$Pref::outlineWidth (0-10) linewidth of outlines
You are free to use this code for whatever project you'd like, as long as you post every change you make on this forum so others can use it too :)
BUGS:
- enviroment mapped models have 'reflective' lines
Please if you like this stuff, contribute and help us with this!
also if you know anything about cell-shading, try to do that too...
in other words:
FRANK BIGNONE: please post your cell shading stuff from DOP here :(
and another word, noone is allowed to submit this as a resource except for me when i think the code is good enough to be posted as a resource :)
About the author
#82
Toby.
04/17/2005 (2:56 pm)
Chris L@ The Random black lines are not only for ATI, but for NVidia. I have a P3, 733 with a Geforece 3 Ti 500 and it has the problem while my P4 2.6 with an ATI 8500 All-In-Wonder has the problem but it's barely visible.Toby.
#84
is it only dts shapes that are messing up for you? For me it was the terrain...
04/17/2005 (3:39 pm)
Interesting... I dont have that problem at all on my Nividia card.. Just the ATI... Is the tree the only place the problem is? cause it might be because of a porrly modeled tree and not because of the algorithm. Probably the algorithm... but gotta make sure first...is it only dts shapes that are messing up for you? For me it was the terrain...
#85

terrain looks ok so far, but not alot of testing

I also see a line on the left string on the crossbow.
04/17/2005 (3:49 pm)
It works good for this shape. I was thinking it may be the poorly made shapes, as i dont have any other objects to test on.
terrain looks ok so far, but not alot of testing

I also see a line on the left string on the crossbow.
#86

if so that is the code doing what it is supposed to do.
It disappears when i get closer because then the polys on the other side can be shown.
04/17/2005 (4:43 pm)
Are you talking about the lines on the right
if so that is the code doing what it is supposed to do.
It disappears when i get closer because then the polys on the other side can be shown.
#87
@Toby - Are you using this algorithm (for outlining) in Ninjasticks?
04/18/2005 (3:36 am)
So, the going theory is that the tree might be messed up?@Toby - Are you using this algorithm (for outlining) in Ninjasticks?
#88
The health pack and ammo are perfect looking as they are not complex.
The best bet for a cel shading type game is to use simple models as the details won't be shown anyways.
Another problem I'm working on is the line width problem for the shapes. If you tell it to draw lines of size 5, it draws it for everything, close and far away. The close objects show up good, but any object that is far away looks like a black glob till you get close enough for the shape to be bigger than the line size. I'm working on finding the distance from to player to the object and reduce the size of the line or elimate outlining on it till you get close enough.
04/18/2005 (4:57 am)
Well to be sure I would need access to some carefully created objects. I don't think the code is the problem as the shape outline code was based off of hehe's tut number 37 I belive and demos created with that are rock solid.The health pack and ammo are perfect looking as they are not complex.
The best bet for a cel shading type game is to use simple models as the details won't be shown anyways.
Another problem I'm working on is the line width problem for the shapes. If you tell it to draw lines of size 5, it draws it for everything, close and far away. The close objects show up good, but any object that is far away looks like a black glob till you get close enough for the shape to be bigger than the line size. I'm working on finding the distance from to player to the object and reduce the size of the line or elimate outlining on it till you get close enough.
#89
I used the tutorial but I altered it quite a bit. Perhaps our solution is a more software based approach... Defining which edges need to be outlined through code and then drawing them at render time.
04/18/2005 (5:24 am)
You should never be using that big of a line unless it's for testing to see them better.. Try line width of 2 or 3.... As for an object to test with... Make a large, highly tesselated sphere... and try that... Maybe Stretch out a couple of the vertices.... Then you can look to see if any of the edges are being drawn on...I used the tutorial but I altered it quite a bit. Perhaps our solution is a more software based approach... Defining which edges need to be outlined through code and then drawing them at render time.
#90
We are using this for Ninjasticks indeed - and for the shapes, use a simple sphere and on my Nvidia I get artifacts, on the ATI I don't.
Toby.
04/18/2005 (6:47 am)
Chris,We are using this for Ninjasticks indeed - and for the shapes, use a simple sphere and on my Nvidia I get artifacts, on the ATI I don't.
Toby.
#91
Can you post a screenie of the sphere please...
Are you rinning the same code I am? THe render code from the top plus my additions (I should note that I had to format my system and lost all of my toon code)
04/18/2005 (7:00 am)
Artifacts on the NVidia... Is it in OpenGl mode or D3D mode? And could you please switch it and see if the artifacts go away. Can you do the same on the ATI system too please?Can you post a screenie of the sphere please...
Are you rinning the same code I am? THe render code from the top plus my additions (I should note that I had to format my system and lost all of my toon code)
#92
Opengl - OK, only a performance hit.
D3D - Crap
Nvidia (From memory):
OpenGl - Present artifacts
D3D - Present artifacts
See for yourself (ATI 8500) - The head is a normal sphere.

Toby.
04/18/2005 (7:29 am)
ATI 8500 All in wonder:Opengl - OK, only a performance hit.
D3D - Crap
Nvidia (From memory):
OpenGl - Present artifacts
D3D - Present artifacts
See for yourself (ATI 8500) - The head is a normal sphere.

Toby.
#93
chazcross@phase9studios.com
04/18/2005 (7:46 am)
Toby can you do me a favor and email me the sphere and the art source file (max, milkshap.. What ever it was made in) so I can do some testing on simple shapes and try to find workarounds when I get home later. I'm not good with the 3d art programs and it would save me lots of time trying to make an object and exporting it.chazcross@phase9studios.com
#95
04/19/2005 (6:52 am)
Does the first post on this topic contain the most updated code or is there a resource for this yet.
#96
The only major difference between the top and the current, is the code mod I posted a few posts up... That contains the terrain outline code. The current issue is with terrain outlining and dts outlines..
04/19/2005 (7:10 am)
It is NOT the most updated code... Toby can give you the most updated code. I had it but I was forced to format my system this weekend and lost that particular engine mod...The only major difference between the top and the current, is the code mod I posted a few posts up... That contains the terrain outline code. The current issue is with terrain outlining and dts outlines..
#97
By the way I am running it on my laptop and it has ATI seems to work fine other then the two problems I said.
04/19/2005 (2:18 pm)
Cool I would like to try out the most up to date. I did what was in the first post and it definitely had some issues but it was really sweet looking. I noticed artifacts poping up every now and then and the frame rate took a big hit.By the way I am running it on my laptop and it has ATI seems to work fine other then the two problems I said.
#98
04/19/2005 (3:45 pm)
Every method so far has issues... I believe this mainly to be because of the different Graphics cards... Rather then finding a drawing method that works for all graphics cards, it would be much more efficient to find an algorithmic approach. Just my opinion of course... There was a .plan (I believe) that involved occlusion to do outlines. I believe it was Stephen Zepp (Sorry if I spelled your name wrong) who wrote it...
#99
in short....
Xavier "eXoDuS" Amado:
Brett Fattori:
There is also thsi resource for a possible solution
www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=2435
04/19/2005 (4:22 pm)
I just found this thread.....www.garagegames.com/mg/forums/result.thread.php?qt=19759in short....
Xavier "eXoDuS" Amado:
Quote:
You will have to do some rendering modifications, namely scale the shape a little bigger than it's size, render it and then render the original shape on top without depth buffer. Or render the one behind with only backfaces.
Brett Fattori:
Quote:Stencil Shadow resource
You could look at the code in the Stencil Shadow resource. It does model edge detection/creation. Modify that to create a glow effect around the model. Check the code that generates the silhouette and then look at the silhouette debug rendering code.
Or, for a really simple effect, take the model and apply a new skin to it (the glow). Scale it up about 1.25 or 1.5 the size of the original. Finally, reverse all the normals in the mesh.
- Brett
There is also thsi resource for a possible solution
www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=2435

Torque Owner Chris Labombard
Premium Preferred