Photogrammetry tutorial 10: an improved method for mid-sized objects

Last December I was granted a sack full of money by the Senatskanzlei Berlin for the 2015 digiS programme. I’ll soon post more about that; what matters for this post is that I promised to mass digitize the rather unwieldy sauropod longbones and girdle bones that reside in the Naturkundemuseum’s Bone Cellar.

BoneCellar_06

Overall, there’s ca. 360 such bones, ranging in size from half a meter to nearly 2.5 meters. As a collection they are, therefore, too heavy to transport around. You can lug one or two or even five up the stairs and into a spacious lab, but not hundreds. Therefore, digitizing has to happen in place, in the confines of the stuffed-full Bone Cellar. And that pretty much excludes the vast majority of scanning methods.

Room to set up a large and sturdy turntable with a background that doesn’t offer any features? Forget it!
Room for a laser scanner? Forget it!
Control over lights? Forget it!
Time to set up each bone in the middle of a lot of free space? Forget it!
Space to wave a held-held scanner around? Maybe, but those that offer a high resolution are quite expensive. Forget it!
Time, space and personnel to take the bones elsewhere? Don’t get me started…….

What was needed was a protocol that allowed quick, high-resolution digitizing in place. And with “quick” I mean quick. My aim is to spend not more than 6 minutes per bone for the entire process, including set-up and all.

Photogrammetry to the rescue!

So here’s how, after some trial& error, I decided to go about it. In general terms, you can copy this process for practically any Big Bone room and similar collection rooms worldwide, and for all kinds of large, heavy objects – provided you are allowed to handle them. Some, like statues and busts, you will not be allowed to stand on their heads. But then, there normally is no need to digitize the underside of a statue ;)

Setting up

The Berlin Bone Cellar has long rows of wooden shelves, and aisles between them that I consider just wide enough to digitize a large bone in them. Therefore, I get help from a preparator to pull out one bone per shelf segment each and  put it on a large piece of Styrofoam in the centre of the aisle. Around each bone I place three scale bars. My colleague Matteo Belvedere has created sets of scale bars of various lengths (10 cm to 50 cm) using coded targets, printed on lightweight but stable foam board.

Coded target: a picture that Photoscan recognizes and can automatically place a marker in the middle of. Basically, an automatically recognizable bull’s eye. If you have two of them on a piece of material that doesn’t bend, and if you know the exact distance between them, you can use that as a scale bar that the software automatically recognizes the ends of. Saves quite some time when you scale your model.

I also prepare a large number of trash bags by cutting their bottoms so they are flat when folded open. I could additionally slit them open lengthwise, but I am too lazy to do that. The last item on the list of auxiliary implements are lots of big, stiff cardboards.

Also, I obviously need a camera. With a fairly wide-angle lens – in my case it’s a Canon 70D with a 10-18 mm lens. And, because the lighting is rather suboptimal in the Bone Cellar, a LED ring light. I bought mine cheap: with a set of high-capacity rechargeable batteries it cost me ~70 € only. I could use a proper Canon ring flash, too (Matteo has one, we ran a test and that worked just fine), but the LED ring light works quite well.

Shooting the image sets

Now is the time to start photographing each bone. Then, they must be flipped over and photographed again – with a twist: I change the background completely for the second set, so that the photogrammetry software cannot find features on the background that link the two sets. This way, I avoid a lot of cumbersome masking even though the background of each photoset is not featureless.

To shoot a photo set, I start by snapping a picture of the specimen tag. Then, I walk around the the first bone, snapping pictures with the camera held low, trying to aim it squarely at the bone at a nearly horizontal angle. Then I do another round, slightly higher up, and finally some overview pics. Or I start with the overview shots and do the near-horizontal round later – doesn’t matter. What counts is making sure I take them all.

I end the round with a shot of the ceiling or the tag again, or whatever – simply something that looks very much different from the bone. That makes sorting photos easier, as the different image will stand out the among the Windows explorer previews.

Flip them bones!

Once all bones have been photographed this way, I put the camera aside (making sure to turn off the LED light, to conserve battery), and start flipping over the bones. Now, it is really important to have help with this process, as I also need to alter the background by placing the trash bags all around and under the bone. The stiff cardboards go against the shelves, so that this part of the background is also altered.

The scale bars must be removed at this step, otherwise features on them will lead to wrong alignment of photos! It doesn’t matter that the next set of photos will be without scales, because it is enough to have scales on a handful of images.

I now take the next set of photos as described above, and for most bones that’s enough. Some, like sauropod tibiae and ulna, tend not to rest stably in two positions roughly 180° apart, but rather in three positions roughly 120° apart. These I need to shoot a third set of. And sometimes I shoot even more sets…..

And then it is time to clear away all the bones and paraphernalia. DONE!

Now I just toss all photos of one bone into one chunk each in Photoscan, and align….. and get models like this one:

3D model

May I introduce MB.R.2636, a left Giraffatitan brancai femur?

Got it?

Well, have you understood what I do? Can you do it yourself?

Thought so….. it is not easy to image how I do this based on the above description alone. Thus, you can find a video on youtube showing me doing this process for one bone (and Matteo helping). Run time is slightly over 5 minutes, so overall I achieved my 6-minute-aim. I didn’t have the stiff cardboards when I did this, so they are missing. As a consequence, I had to mask the second set of photographs. But other than that the video shows exactly how I do things.

Posted in 3D modeling, Digitizing, Dinosauria, Giraffatitan, How to, MfN Berlin, photogrammetry, Sauropoda, Sauropodomorpha | 2 Comments

The BBC’s awesome ‘Planet Dinosaur’ now in 3D on Blu-ray

A few years ago I had the pleasure to work with people from Jellyfish Pictures on what was to become known as the BBC’s Planet Dinosaur series (wikipedia). And as opposed to the all-too-normal process, where the producers have basically formed a complete mental picture of how the film is to be made, where the artists have already build rigged 3D models and have animated them, where the storyline is completely finished BEFORE experts are brought in (a process that invariably leads to frustration all around)  Jellyfish contacted dinosaur experts worldwide in advance, when only rough drafts had been made. Someone among the large number of people they contacted suggested they should also talk to me. They did, and it was a fun process that (I hope) improved the final result a bit.

clapper

The really great thing was that they would send me videos they wanted my comments on, then we had a phone conference with the artists (! – they got my feedback unfiltered) while running Cinesync, a program that lets several people at different computer see and interact with the same video. I could stop and advance or reverse the video as I pleased and draw and type on the screen, as could everyone else – very helpful! The videos showed early drafts of 3D models in simple walking or running cycles, and we talked through any issues very thoroughly. A great bunch of people, who didn’t let their egos get tied to positions, so that I was free to simply say “that’s wrong, you need to fix this, because……” instead of having to pussyfoot around issues. They clearly valued the time I put in, and wanted to make the most of it.

One of the things I noted early on was pronated hands on all theropods, i.e. they had their palms facing down, not in. That’s a common error, and it was only over the last decade that some colleagues and I, especially Matt Bonnan and Phil Senter with a seminal paper, managed to raise awareness for the non-pronated state of most dinosaur hands. Dave Hone wrote a wonderful post on his blog in 2009, clarifying that theropods were clappers, not slappers. In any case, I told the assembled artists that a slapper-handed theropod reminded me of a little old lady carrying her large bag by gripping it with both hands from above. Laughter all around and the hands got altered – see above!

clapper

A fun thing was that Jellyfish contacted me about theropods and ornithischians initially. When I told them I was more of a sauropod person, being employed in the sauropod research unit FOR 533, they also started showing me sauropod material and asked me on their planned storyline, despite having sauropod people on board already. A smart decision, as they got more info, and for free.

A specific topic I remember well was baby sauropods. They had this nice idea of them hatching and running around being cute, which they implemented very well – see above. However, I also suggested that baby sauropods were, essentially, meals on stilts for large theropods. And, to their great credit, although a segment showing huge ugly beasts chugging down cute tiny babies wasn’t really something that sells very well to a general audience, they did put it in! Hooray!!!!!!!

clapper

gobble, gobble,……

clapper

yummy!!!!!

I must say I was very much surprised that they actually went and put a scene in that clearly was not going to be a viewers’ favourite, just because it was science’s best reconstruction of the actual life history of dinosaurs!

In the end, I greatly enjoyed working with the Jellyfish and BBC people, and I liked their final product even more! Below is another shot from it that shows off the wonderful work they did – which you can now finally buy in the 3D version on Blu-ray. Yes, that’s what this post really is about, although I relished the chance to laud the great team and the great result again: you can buy Planet Dinosaur 3D now. Go do it – it is totally worth it! I haven’t seen it yet, but I have seen a few out-takes using these weird red-green glasses, and it does make the entire thing look quite a bit more alive – even more alive I should say. Here’s the link to the shop (US).  It’s only ~25 bucks, for a really great dino documentary.

clapper

Posted in 3D modeling, Dinosaur models, Dinosauria | 7 Comments

Australia!

Yesterday, after a rather excruciatingly long flight, I arrived in Armidale. I’m spending two weeks here with the FEAR Lab, doing some modelling work. I’ll obviously report on that, but for now I just want to dump a bunch of photos on you.

Sydney

This is a shot from my stopover in Sydney, where I went through immigration and – surprise! had to stand in line again for customs. Well, quarantine….. which makes a lot of sense, seeing how Australia has a flora and fauna totally separated from much of the rest of the world. Therefore, they do a quick check of what people and luggage they want to check in detail at the customs check, and that caused a line to form. This little incident made being on a totally different continent quite real for me.

Oh, and as you can see, all of Sydney easily fits under an A-380 wing ;)

Sydney

Unsurprisingly, a totally different fauna also means totally different birds. So far, I’ve only seen one species of bird that I’ve seen before outside zoos – unsurprisingly the European starling. Other than that Armidale is full of Australian magpies (Cracticus tibicen), above on the antenna of my hotel, who make their ubiquitous presence very well heard all over town. Being used to corvids making “koww” of “cawh” sounds, it is weird to hear a bird that looks just like one imitating a flute, from which they got their German name of Flötenvogel (flute bird).

Sydney

I also saw several kookaburras, Dacelo novaeguineae. This one flew in and sat on the playground equipment in a park while I stood a few meters away.

Sydney

A red wattlebird (Anthochaera carunculata) – I heard a lot of them, and saw several.

and lots more – for two hours of walking around town(!) I photographed a surprisingly large number of taxa.

Posted in Aves, Maniraptora, Theropoda | 2 Comments

Theropod Thursday 53: Snow White

Last Monday, work took me to the Weltvogelpark Walsrode (wikipedia). It is really weird that I haven’t been there before, and I was determined to make the most of this visit, despite a so-so weather forecast. The Weltvogelpark is a huge one, the largest bird park in the world both in number of species and land area (claims wikipedia), and it has only birds (no, not even guinea pigs somewhere), as opposed to many other bird parks. You’ll get Theropod Thursday-ed to death here with photos I took there, but today I want to focus on a handful of birds I saw that that have one thing in common: they are white.

Let me warm you up with a bird that is mostly white – in the photo below I managed to “hide” the dark head in an underexposed background ;)

Madagascar sacred ibis

This is Threskiornis bernieri, the Madagascar or Malagasy sacred ibis. Looks a bit ruffled, which is not surprising. Monday was a stormy and rainy day, with some hail thrown in for fun, too. The next bird doesn’t look ruffled, and doesn’t have any non-white feathers.

white

Can you guess the species? Or at least the genus?

OK, more info may help you……

white

recognize it now? It good old Corvus corone, the Carrion crow! An albino one, to by sure, as the red eyes show. Real albinism, a total lack of the pigment melanin, is much rarer than just a reduced amount of melanin, a condition called leucism. Leucistic animals (including humans) typically have enough pigment in the iris to make it not totally translucent, and so we do not see the blood vessel rich retina.

white

Yep, an albino! And if this bird doesn’t deserve the name Snow White, maybe this one does:

white

white

white

As you can see, this peacock (Pavo cristatus, I guess from the head shape and the other animals running free in Walsrode) is leucistic, but not an albino. The eyes are black.

So that’s two really rare things here: an albino crow and a leucistic peacock – in one place. But Walsrode has much more to offer than just freaks (as I threatened above). Today, I’ll wrap up the post on white birds with one that is completely white except for its beak and the tips of a handful of feathers: the coscoroba swan (Coscoroba coscoroba). Walsrode has a pair, and while one of the was sitting on the nest, the other was quite a show-off. Clearly, the intent was to keep us sufficiently impressed so that we would stay away from the nest. The coscoroba isn’t the biggest of swans, but its ego is certainly not too small.

white

white

white

white

 

 

 

Posted in Aves, Dinopics, Dinosauria, Theropoda, Weltvogelpark Walsrode, Zoos | 1 Comment

Theropod Thursday 52: pulling a leg! Erhm….rather: pulling a tail!

hiding_02

Hungry I am. An aardwolf (Proteles cristata), too. A hungry aardwolf. Yep, that’s it. A hungry aardwolf. Now get me my food. Now!

This is the elusive critter that David Hone, my esteemed colleague with plenty of zoo experience, and I totally missed seeing during our legendary winter visit. Legendary for me, as it made me aware of how much cooler a zoo the Tierpark is than I was previously taking note of. Dave kept repeating “I’ve never seen this animal before!” The aardwolf was the one animal that we really wanted to see but didn’t. And on my many other visits to the Tierpark – I place I practically live at during summer weekends – the aardwolf stayed elusive. I did see it, but generally just as a furry blur popping out of one hole in the ground and into another one.

So imagine my delight when the Tierpark, under the new director Dr. Knieriem, started announcing the feeding times of some of the animals, and included not only the echidna but also the aardwolf on this list! Obviously, I dragged my children there at the appointed time, 2:15 p.m. And lo and behold, the elusive aardwolf put in an appearance! He came out of his hiding place when fodder was presented and greedily gobbled it up.

hiding_02

hiding_01

But, waitamoment!” you say. “What’s this to do with Theropod Thursday?

Well, here goes:

hiding_02

A hooded crow (Corvus cornix) is paying a visit. And not a polite one, as we will quickly see.

Aardwolves feed mostly on termites, so the Tierpark feeds them a pretty yucky looking protein-rich slurry. And believe me, crows love anything protein-rich. And crows being crovids, some of the most inventive birds, they come up with ways to make you, me, and aardwolves involuntarily share that yummy protein cocktail.

hiding_02

grab……..

hiding_02

and pull!

Hm, ain’t nothing happening. Well, let’s do this again.

hiding_02

Ouch!

Now, corvids have a pathological obsession with tail pulling. They just can’t help themselves! See here. If it’s got a tail, that tail’s gotta be pulled!

Still no effect? Here we go again!

aardwolf

harder!

aarwolf

Oh wow, finally there’s a reaction!

aardwolf

aardwolf

and so, finally, the poor crow that risked life and wing got the deserved go at the trough!

To be honest, I don’t believe the aardwolf minded the tail pulling very much. He simply had had his fill, and thus would have wandered off anyways.
To end this on a more theropodish than mammalian way, here’s a Hooded crow proudly strutting around.

aardwolf

Posted in Aves, Dinosauria, Mammal pic, Mammalia, Maniraptora, Theropoda, Zoos | 1 Comment

Photogrammetry tutorial 9: Quick and dirty!

Over the course of previous tutorials and in my paper with Oliver Wings I’ve given quite a bit of advice on how to photogrammetrize objects properly. Today, I’ll address the other end of the spectrum: how to approach photogrammetry unprepared and without proper equipment. Oliver and I mentioned a minimum kit in our paper, but what if you’re really stuck with just a digital camera? What can you do with everyday materials to improve your results, and how do you approach the entire thing?

First of all, you really need to know how to handle your camera. If you have a tripod and control over the lighting, it is easy to take in-focus and well-lit images. But if the light is low, comes from one direction, and you must hand-hold your camera, then it becomes important that you know with which lens at what zoom setting you can reliably take good photographs with what ISO settings. There’s one thing that will help you a lot that usually I’d strongly recommend against: use flash! Yes, you read that right: it is better to use a flash, even your camera’s internal flash, than take blurry photos. (If your camera doesn’t have an internal flash, then it is a top notch DSLR and I can’t believe you don’t have a whole shitload of equipment with you). There’s an important thing to do when you use your internal flash, but I’ll save that for later when I talk about the process of photographing a specimen.

One thing to remember is that your zoom lenses will allow shorter exposure times / lower ISO / smaller apertures if you use them at a low extension length. So do consider stepping closer to the specimen rather than zooming in. As long as you do not go into the ultra-wide angles or use a fisheye, your models should turn out just fine. And if your modelling software happens to be Agisoft Photoscan Pro, you can even use a fisheye lens, as the program has a feature that allows dealing with fisheye distortion.

Another issue with q&d photogrammetry is usually the inability to use a turntable or optimize the background. Normally, you’ll want to use a background that doesn’t offer any features for the software to detect. However, given the lower quality of images you are wont to achieve, using a feature-rich background can actually be a boon! Normally, you’ll always want to aim for one-chunk model creation, where you toss all images into one chunk, even if the specimen was moved versus the background in between. Instead, now try to use a highly structured background, such as a colour ad newspaper page! And once you’ve finished shooting the specimen from one side and move it by flipping it over, use a different feature-rich background! This way, the background can be included in model creation and thus help with alignment, but doesn’t cause images to align based on the background that were taken with different specimen positions.

Let me explain step by step:

  1. Put specimen on a double-wide newspaper page with colour adverts, or some other crazy stuff. Persian rug or whatever.
  2. Clean, as far as possible, the area around or use additional newspaper etc. to hide anything that would be in the picture.
  3. Shoot images from all around.
  4. Remove specimen and all stuff you used to cover the table.
  5. Place different newspaper – maybe even something totally different like a tablecloth.
  6. Return specimen in different position (flipped over).
  7. Shoot images from all around.

Now, you can use multi-chunk alignment with points-based chunk alignment, as nothing but the specimen is there to make the chunks match each other. Granted, it may not work, but you can always resort to manually placed markers.

OK, now is the time to say that bit about using the internal flash. Obviously, in an ideal world, you have brought a ring light (usually LED) or ring flash, but let’s assume that you haven’t. If you point your camera at the specimen you’ll usually hold it upside up, so that the flash is above the lens. And in normal conditions, whether outside or indoors, the majority of the light will come form above. This means that you naturally get a darker, less-lit underside of the specimen, and that you add to this effect by having the flash light slightly from above, too. So, fix this: flip your camera upside-down! It sounds stupid, and it can be cumbersome, but having the flash under the lens gives you an extra 10% or so of surface you capture well-lit, right where it matters: on the outward edge of the specimen, where you want to find many good points to match one chunk to another.

What else? For one thing, do try out unusual camera programmes! Many DSRLs and even cheap point&shooters offer HDR modes. These modes take several images at different ISOs (usually) or exposures (better but rarer), and combine them into one image that is more evenly lit than any normal photo. As this post direly needs a photo now, here’s one – a HDR mode photo I took with my Canon EOS 650D of the AMNH’s Tyrannosaurus mount.

AMNH T. rex

AMNH T. rex

Oh well, it’s a cool mount, so two photographs instead of one ;) Notice how there are no black shadows and no white-outs? That’s the power of automated HDR. Sadly, the 650D does this via ISO variation, and that includes high ISO values, leading to a lot of noise in the image. But still: these images align and produce dense point clouds much better in Photoscan than normal, hand-held ones would. So, no tripod? Try the HDR mode! If you run Magic Lantern on your camera, you can do better, varying the exposure, not the ISO, too.

Anything else? Consider making a makeshift turntable from a sandbox or so for small specimens, or just using a smooth board on a smooth table. Use white pieces of paper or white shirts to reflect light onto specimens. And so on – happy digitizing! :)

 

Posted in AMNH, Digitizing, Dinopics, Dinosauria, How to, photogrammetry, Theropoda, Tyrannosauridae, Tyrannosaurus | 5 Comments

Photogrammetry tutorial 8: scaling “with hindsight”

Recently, I mentioned in passing that I took some photographs of specimens at the AMNH in New York so that I could better scale models I had calculated from photographs taken during my previous visit. Here’s how that worked out, with a step-by-step explanation.

Here’s one of the models again, as calculated before, without a proper large scale bar.

oviporn

If your name is Jaime H., click for a larger version ;)

This is a model of the Khaan mckennai holotype IGM 100/1127. It has only some 17.5 million polygons. Still, a nice model! Obviously, given the advances in the photogrammetry software that happened since I previously calculated the models, I could simply have re-run them from scratch, just with the new images added. In fact, I am doing that also, to compare the results. But with a calculation time of 183 hours for the alignment and 75 hours for dense point cloud generation – and that is for ONE of the models – it is a major task. If I can avoid it, that’s an option nice to have. I just recalculated on of my models, and that has 279 million points in the dense point cloud. So, “only” 17.5 million here, but a model that I certainly do not wish to re-calculate, as it is good enough for the vast majority of purposes.

So, how to proceed?
First of all, I created a new chunk in the project file, and added my new photographs with the big scale bars into it. I ran alignment, obviously at ‘High’ accuracy. Then, I added markers on my scale bars in the photographs:

markers

Obviously, the surroundings have changed. And if you look closely you will note that the specimen itself has changed a bit, due to further preparation work. However, the by far largest part of the area photographed has stayed the same. Therefore, it is now possible to align the two chunks to each other via points. And then I merged the chunks, which retained the already-calculated model.

markers

This is the chunk with the old photos, showing the alignment. And here’s the two chunks aligned and merged:

markers

Note that I set the alignment photos to “inactive” so that they are shown in a different colour.

Now, I just updated the merged chunks using two scale bars created from the four markers – and done!

OK, that’s it! And now, to tease certain colleague a bit more, another view of the model. This time without colour, so that the high quality is more apparent:

oviporn2

 

 

 

 

Posted in AMNH, Digitizing, Dinopics, Dinosaur models, Dinosauria, Khaan, Maniraptora, Oviraptorosauridae, photogrammetry, Theropoda | 7 Comments