Blocking the Animatic

When I started blocking out the animatic this term it all started to become very real in both exciting and scary ways! Seeing the shots from the animatic come together a lot easier than I imagined was very satisfying, this is a big jump in the production where things are starting to come to life. However, I knew getting to this point meant I'd have to start tackling how to get this final style working and animated well when I still had so much to learn.

Like with everything so far, there's only so much I could research and actually doing it was the only way to fully understand how to get it working, so I have focused on enjoying seeing these scenes come to life and taking things on as I go.

As there is only one primary environment in my film, the rest being more abstract and lose in design, I was able to just focus on this room and plan out the shots from there. From my first term of this course, I completed a tower of boxes as an experiment in getting to grips with Maya, I had also used this tower to play with the controls in MASH in the previous term. This tower was a perfectly good asset to base my box room around. Duplicating the tower I suddenly had two sides of the room complete. I added the walls and door, then started filling in a few more boxes and objects to match shots from the animatic.

Because my animatic is so lighting-based, I also began to experiment with lighting here. I ran through a few options of what could be used in this first shot with the door opening. I tried a spotlight, area light and just using the skydome but I settled with using a mixture of area light, skydome and directional light.

The whole environment:

Apart from a few well-placed cylinders, I tried to keep modelling any other objects n the scene to a minimum as I know it would take a while with my lack of experience, and I may add more 2D objects later in production. But I would be needed a 3D model of an open box for the character to be looking through. This took a while to perfect but I am happy with the final thing! I even duplicated it to use in other places within the scene.

Blocking with the Character

As I was still struggling to get the rig sorted for a while the first character blocking I did was with a non-movable T-Posing version of the character. Luckily this was after I got the textures onto the model so that really helped as I continued to do render tests and see how the model's colours worked in conjunction with the box's colours.

MASH

I went through a journey with MASH this term.

As I had been using the blocked-out towers for my previous experiments with MASH the last term, I continued to use them as my main way of building environments. However, after building the box mountains I realised this was detrimental when I got to adding physics to the scenes. When adding the dynamic node, the node that controls the physics, the separate boxes didn't move as they should do in gravity, but rather they moved together as a unit bouncing off tower from tower. This is because they are going from the MASH network created from that mesh.

So this box mountain below, unfortunately, had to be scraped so I could get something working that actually looked like it had fallen down.

So I started from scratch with MASH and, as reluctant as I was to abandon my box towers, I am glad this gave me the chance to learn it more thoroughly. By creating a scene that based the mash network on only one box, I was able to free up a lot more abilities than I had before.

From this, I particularly made use of the Random node which helps you randomise your instances into different sizes, angles, and heights so that you don't have any recognisable patterns. The patterns were a definite problem when using the towers as there were so many distinguishable parts and the random node only moved around the same distinguishable parts.

The issue with the tower patterns is visible here:

Finally using the Dynamics node on these new towers was so much more effective and really reinstated my faith in MASH. I was able to create small box mountains with the result of gravity and use the camera angle to make them seem larger or smaller.

Here is a Playblast of the MASH Dynamics in action:

Thoughts on Blocking out in the Pipeline

I think this is the step that is most important to do as soon as possible within the 3D pipeline. It really helped kick me forward into producing the rest of the 3D. By having the blocked-out environment, without really thinking I was already working on lighting and camera angles before I even planned to! It was a great motivator to continue on with texturing and getting it to go from 'blocked out' to final.

A big help with this process was learning the brilliance of MASH, it is certainly a tool I would like to come back to as it feels like I only scratched the surface of what is possible with it. It really demystified so many questions I had about how things were done in 3D and revealed ways that I could realise my 2D vision. Kelly Clayton was a great help here, showing me how to make my first MASH network. Learning to ask for help from my coursemates has been something I've gotten better at over the course of the year.

Rigging

Trails and tribulations of the Advanced Skeleton Plugin

https://www.animationstudios.com.au/advanced-skeleton

In an attempt to lessen the load and skip what I'd heard to be a long process of rigging a character by hand through Maya's standard controls, I thought I'd used an automated plugin programmed to build it for me. I had heard of this plugin through Lipin Murali and Isaac Boardman as they had used this specifically for Rocket Dog. Isaac sent me the link and I got it set up on my PC nice and quickly.

There were some good tutorials on the Advanced Skeleton website and at https://www.youtube.com/user/AdvancedSkeleton but I found a more detailed step-by-step one on YouTube from https://www.youtube.com/c/AnimatorArtistLife which I followed.

The majority of the process was very straightforward, lining up half a standard human joint skeleton to the model, and adjusting the length and position of joints when needed. Once everything was lined up, including arms, fingers and root positioning, the advanced skeleton builds up the other half mirroring the joints onto the other side of the model.

After this stage the Skin Cage was calculated; this creates an area around the character that will be what moves when the joints are moved. This was followed by adjusting the skins so that there was nothing overlapping, articulating focusing on the finger area as them being so close together can cause problems.

An important step was adjusting the joint controls. As the model rig was generated so were the controls and some had generated inside the model making it hard to select and use them easily. Luckily Advance Skeleton has a tool to scale up the controls and adjust this when needed.

To help give extra control of the rig and how the movement will look, there is a step called Painting Skin weights. This is the process of adding weight to the correct areas, for example, the shoulder area is one that is likely to look warped when it is moved in the rig. Doing this helps control how this warp will look.

With that, the body was complete and bound to the model.

Moving on to the face there were many more steps for the build to be completed. However, I noticed there were issues with this setup from the beginning.

Above is what the set-up is supposed to look like. And below is what I was experiencing.

With one of the first steps being to select edges for where the eyelid would be going, I noticed it was not becoming a red blocked-out shape upon finishing it, as shown in the model above. I redid this step a few times just in case there was something I missed but I kept seeing the same result. Thinking it could just be a visual glitch and the system could still work for the overall build, I continued on to the mouth area where I also did not see a red area appear. With the same logic, I continued on and placed the dots around the eyebrows, jaw, cheek, chin and nose as the tutorials instructed. Another worrying sign happened when I did the steps for the forehead and instead of creating that green blocked shape between the vertices I selected, brightly visible on the model above, it created confused-looking green lines. It was definitely looking unhopeful for this step in the rigging process.

Once this was complete I should, in theory, be able to press Build Face Skeleton and it would automate the build, taking some time to go through some expressions and create the rig. Unsurprisingly this did not work. What happened after I clicked for the build to start was a very quick loading time, unlike the tutorials, and then a control appeared similar to the tutorials but only two controls within, one for each eyeball. This eyeball control worked which was great, but unfortunately, nothing else on the face had a control.

I tried a few things to get this fixed. Going over the whole process multiple times didn't get me anywhere so there was nothing I missed. I tried going through it with the unsymmetrical face settings, so doing the selections for the whole face rather than just the one side. This did change the result. I also tried going through the build one part at a time, I feature of the Advance Skeleton specifically there for if the automatic face build didn't work. This again got me as far as the eyeballs before not letting me get any further.

The Advanced Skeleton is specific in saying the model would work best if entirely symmetrical. My model had a slightly different typology on the face due to the hair blocking a larger area of the forehead. In my many attempts to fix the issue, I experimented by mirroring the face topology and this still didn't work. The plugin itself was completely up to date so I really struggled for a while to understand the issue.

I had decided to use the Advanced Skeleton Plugin as a way to overcome the large and long task of building the rig by hand, as I had limited time to complete many steps this term, this seemed like the best solution. Unfortunately, this actually ended up costing me a few days of trial and error and I definitely should have moved on to an alternative method sooner.

Blend Shapes

My solution to the face rig issue was Blend Shapes. It is something I was somewhat aware of from my research but I had dismissed them as something that could potentially take too long due to the idea that you would have to hand sculpt each expression. Looking at it in comparison to rigging the face by hand, however, I could see that this was the better option. It was a choice between trying something I had never done before, going through a 5-hour tutorial and hoping it would pay off, or sculpting in a way I was familiar with and simply changing out the expressions shown clearly in a few 15-minute tutorials. It also helped that I knew the exact expressions I would be needing from what I'd already planned out in the animatic, so I didn't need to make more than what I needed.

Another good thing about using Blend Shapes is that you can mix up your expressions and use a little bit of one to change another slightly. For example, I could use 0.5 of the surprised face and 0.5 of the scared/sad face to show a more shocked expression.

With expressions done that just left the eyeball movement to be rigged and I already knew a method that without a doubt worked for that. So the Advanced Skeleton didn't completely fail me with the face.

Closing Thoughts on the Rigging Step in the Pipeline

Whilst my experience didn't turn out to be the most efficient I think this pipeline of the automatic plugins and blend shapes is an interesting one.

Overall the Advanced Skeleton plugin was extremely effective at giving me a perfectly working body rig. The body has some great natural movement and has the option to use both FK and IK forms of movement. IK legs help the knees and leg joints move naturally and with the options to bend the arms with FK moves or an IK blend mode it really gives you the freedom to animate however you'd like.

For me, with my short film animatic all worked out with the knowledge of exactly what expressions I needed, the blend shapes were a perfect option to save the face rigging issue. If this were a rig for a longer feature or a TV show, however, I believe there would have needed to be a lot more options within the expression library. Even then, the blend shape seems to be a very convenient feature that could be applied to more items than just expressions so a very exciting thing to learn about.

Texturing The Character

The texturing process of the pipeline definitely had its up and downs. It was mostly downs until I finally got to painting on the stylised texture I wanted. To get there, however, there were many setbacks in the process.

UV Mapping

Firstly, I had to start with UV Mapping. This was a process of looking at the unwrapped map of the model and cutting seams for where each part's texture would change. So the skin is on one level, the hair on another, clothing, eyes etc. This was a relatively easy task, I had some great guidance from Kelly Clayton, advising me on things such as how the seams on the clothing parts of the model can just match how they would do on a normal item of clothing. This would then make things look more natural when the final textures were on there.

Baking the Model

After the UV mapping was complete and saved to different texture shaders according to what was going where it was then time to attempt to get the model into Adobe Substance Painter. This is where I had some trouble. You see in order to get the most out of the higher poly sculpted model you have to perform a step known as baking. This bakes the details of the High Poly onto the Low Poly model you have UV Mapped, therefore allowing you to paint your texture onto the right details and have a Normal Map of this you later plug into Maya. A Normal Map is a way of showing the bumps and dents in a 3D object with just a 2d texture.

This is where the setbacks started to raise as I had to spend a good long while figuring out why my model wouldn’t bake correctly. There were many possibilities I went through with the help of Kelly. The main idea was to separate all the main parts of the High Poly model so that it was on clearer geometry, so I got to work doing that. Luckily it wasn’t too difficult as I had a ZBrush file which still contained the parts of the model on separate sub-tools.

There was also another option to solve the baking issue by simply baking it in ZBrush, which did not turn out as simply as first thought though as it required the long repetition of baking each item separately rather than the quick way Substance Painter would get it to work. So I continued work on correcting the model in Maya with the separate High Poly parts from ZBrush. There was an issue here as there had been alternations correcting some positioning on the arms and a number of other corrections. After things had been lined up again and more work done on connecting inconsistencies between the High Poly and Low Poly model, I continued to try and bake the model again in Substance Painter and with the help of Kelly Clayton, it was finally able to be worked on!

We discovered the best way for the back to work was for both the hair and the eyeballs to be moved off the character to be in the air beside it. This made for a difficult time when working on the eyes as I wouldn’t quite be sure that the irises were the right size until I put them back on the model after texturing but at least it was able to work at all!

There were still a few issues that seemed to not line up however I had spent so much time on this stage that I made the decision to move on rather than waste more time fixing small details.

Texturing in Substance Painter

Finally getting to paint the model was so satisfying and it felt so good to get back to something I was a bit more experienced at with digital painting, however, I had never painted onto a 3D object before so getting to grips with that and the new software was definitely the most fun I have had so far in the modelling pipeline!

After a bit of experimenting to understand the tools, layers and masks with the help of multiple YouTube tutorials, I started with one fill layer to get the base colour down. I turn used a paint layer to start adding the variations of colours on the face. As I am going for a style similar to Arcane, stylistic with painterly shading based on real features, I wanted to start with a good base to keep the face looking natural enough through the painterly style.

I used these references to paint the base colour:

Adding these colours then adjusting the layer opacity and a blur filter to blend more with the general skin tone was a great point to start adding more colour to.

For the extra painterly shading I wanted to do I used these reference images as inspiration for shading positions and how far to push the stylisation.

I held back a little with this as it’s hard to say from these references what is texture on the model is and what its cleverly created lighting. I realised after completing this there may have been a way to find what the unlit models from Arcane looked like if I searched deep enough into Art Station, but I am still happy with the outcome and working from these references was working from something closer to what initially inspired me in the first place, so this felt more enjoyable.

The main areas I focused on were around the eyes. I knew this was an area I wanted to add a lot of stylistic texture shading as I feel it adds depth to the character, showing age and exhaustion. I wanted these bags under the eyes to show the strain this character goes through, it indicates his exhaustion before even starting the project. With this design, there’s already this sympathy and empathy some might have for a character who might work full time, be exhausted when they come home, too run down to do self-care acts of keeping their space clear and clean. This was an important aspect I wanted to get across in my character design, using the eye bags and hunched-over frame to indicate perhaps an office job or just a general depressed look.

As well as making use of the various brushes Substance Painter had available, I also used some of the filters to give more painterly looks. Amongst the brushes and textures they had available, there were specific materials that helped me give the clothing the right look. There was a knit texture that I used as a filter to draw on the jumper. There was also a brush that was able to create a seam line which is what I used for the jeans.

Other than the visual development process, this Substance Painter experience was by far my favourite part of the 3D pipeline so far. Despite the struggle with the baking initially, the gratification of finally seeing the model with the colour and style I have been imagining this whole time come together and work was worth it all.

Here is the model before a few changes:

Here is the final model texture with the Toon Shading effect:

Tutorials used:

https://www.youtube.com/watch?v=RQ-hRk0WHJ8&t=10s

Stylisation:

https://www.youtube.com/watch?v=ewJUHANsiFI

https://www.youtube.com/watch?v=h8llGEKIQT0

Face:

https://www.youtube.com/watch?v=VcH47dl1xWg

https://www.youtube.com/watch?v=V_E1ARR_AAg