Bevel shader
With Forward rendering and default sharp-edged mesh, there isn't enough data for any shader to give edges a bevelled look. So I ended up creating 2 ways for non-traditional Beveling.
1. Low poly bevel edge
This one is very simple, can work with any shader, but requires making modifications to the mesh form. Adding a single face per edge. I write about it in more detail in this article.
2. Bevel Edge Shader
This method is the topic of this article. It can be achieved with a few clicks inside Unity to modify the mesh (using Playtime Painter).
Recipe
Simulating the Bevel effect in Shader requires additional normal vectors for each vertex. We need to have the following data in each vertex:
Smooth normal
I calculate this vector as if all vertices were shared between triangles.
Sharp Normal
Vector of the triangle, since all triangles are made of unique vertices for this to work.
2 Normal vectors,
One per triangle line. We only care about the two attached to the vertex.
Edge detection
One float3 is needed to package edge detection (will yield a value of one during interpolation).
Point count and triangle count remains the same. In the Profile image, you can see 3 LineNormals. I replace one of them with Sharp Normal, but which one depends on in which position the given vertex is in a triangle. And the regular NORMAL is the smooth normal. In my mesh editor, I allow myself to select independently if points should have unique vertices and if those vertices should be smoothed.
"Smooth All" should be clicked in Point Editing mode to mark all vertices as smoothed for the Shader to work. And "All Unique" because every triangle has to consist of unique vertices. Mesh Editor itself doesn't know about Bevel shader, so unfortunately it will not provide any hints on how to do it right. But some hints are provided through the Shader tags. For example, it should automatically select the correct Packaging profile.
My Playtime Painter can be downloaded from the public GitHub, buying it in Asset Store is optional.
Sidenote
Edge Strength, Atlas Texture and Color doesn't relate to this particular effect, but I believe it will make an excellent synergy - I use coloured edge to hide the seams. (Added bumped Bevel Shader below)
Update (2/12/2018)
The main downside was those nasty edges which ended up looking in the opposite direction from the camera. Quite often, they are lit differently and become very visible. But the problem became the solution: comparing them to camera view direction and clipping is easy:
Sometimes looked weird so I ended up not using it.
Video
Examples
For testing, I downloaded some Low-Poly models from Unity Asset Store:
Code
The code for Normal calculation is pretty lite:
inline float3 DetectSmoothEdge(float3 edge, float3 junkNorm, float3 sharpNorm, float3 edge0, float3 edge1, float3 edge2) {
edge = max(0, edge - 0.965) * 28;
float allof = edge.r + edge.g + edge.b;
float border = min(1, allof);
float3 edgeN = edge0*edge.r + edge1*edge.g + edge2*edge.b;
float junk = min(1, (edge.g*edge.b + edge.r*edge.b + edge.r*edge.g)*2);
return normalize((sharpNorm*(1 - border)+ border*edgeN)*(1 - junk) + junk*(junkNorm));
}
Below is a version of this Bevel shader that uses a Bump Map. I use edge detection to mask some seams with vertex colour. Vertex colour Alpha and Texture alpha is used as smoothness - that is why the second image has different gloss, but textures and meshes in both examples are the same. The light model (formula for reflected light & brightness) is also a bit different - just an artistic choice, not related to beveling.
