Cameras and Photosensitive Detectors I: How Do They Work?

00:00:11;23 Hi, my name is Nico Stuurman and I'll be talking today about
00:00:15;27 detectors. So for many, many centuries, we haven't really
00:00:19;21 seen detectors as an important part of the microscope.
00:00:23;21 And that is because we used to just simply look through the
00:00:27;20 eyepiece, observe what we saw, and then document
00:00:31;07 that through these beautiful drawings. Like this one here made by
00:00:35;24 Ramon y Cajal. So, with the advent of cameras, that whole
00:00:42;14 picture changed. And we can now use a modern microscope
00:00:47;27 and the associated cameras to document in a more
00:00:53;00 objective way, what is being observed through the microscope.
00:00:59;02 And an important part of that is also the computer, since in the end,
00:01:04;01 the data from that camera will end up in the computer
00:01:07;22 and form the real image. So not only can we now
00:01:11;04 make much more reliable measurements, we can also do things
00:01:15;06 like shown here, we can make movies, time-lapse
00:01:20;16 sequences, of living specimens that we observe through the
00:01:26;12 microscope. So here you see a cell going through mitosis.
00:01:30;29 Labeled in read for histones/DNA, and in green for microtubules.
00:01:36;19 So, in this talk, I will discuss two different types of detectors.
00:01:44;10 One are single point detectors, so they measure only
00:01:48;18 a single point of light at a time. And the other are these multi-point
00:01:52;26 detectors, so those are detectors that measure multiple points
00:01:57;11 simultaneously. Here are the cameras. Here you see an
00:02:01;05 example of a single point detector and this photo multiplier
00:02:04;02 tube, and here is a camera. Now all these types of detectors
00:02:09;13 share the same principle. So they take in light in the form
00:02:14;22 of photons, and they convert that into electrons. Through the
00:02:20;16 photoelectric effect. At a certain point, the charge, the number of
00:02:26;26 electrons is converted into a voltage. And then that voltage is
00:02:32;08 subsequently converted into a digital number that is being
00:02:35;29 transported into the computer. And then the computer then uses the
00:02:39;09 digital number to build up the image. So there's always the
00:02:43;18 sequence that we go from a charge to a voltage to
00:02:47;08 a digital number. So first, single point detectors. So what's
00:02:52;25 happening there is that we have a beam of light
00:02:56;23 for instance, going through an object that we image with the
00:03:01;08 microscope. And then that beam of light, in the end, makes
00:03:04;11 its way onto the detector. We then scan the sample, for instance,
00:03:09;15 a very simple way is scanning the stage in x and y.
00:03:13;12 Or we can actually scan the beam of light through the sample.
00:03:18;16 And at the detector, we measure at every point in time
00:03:22;20 how much light is falling onto it. We then convert that
00:03:27;11 into a digital number, that digital number goes into the
00:03:31;20 computer, we get this sequence here of numbers. And
00:03:36;01 that is used to build up the image. So what's important with a
00:03:42;10 single point scanner like this is that they're fast. Because
00:03:46;00 since we measure each point individually, it takes a lot of
00:03:49;13 time to build up the whole image. So we need those measurements
00:03:53;21 very, very quickly. Now multiple point detectors, cameras on the
00:03:59;15 other hand, we use a lens to project the image straight
00:04:04;29 onto this multipoint array. So we get an image formed
00:04:10;16 onto the camera itself. But from then on, it's again more or less
00:04:15;29 the same principle. All those photosensitive elements
00:04:19;21 have built up charge, and at a certain point in time, the charge
00:04:23;01 there is converted into a voltage, and in the end, it's
00:04:27;03 converted into a sequence of numbers that is being sent
00:04:31;10 to the computer, and the computer uses that to build
00:04:33;28 up the image. Okay, I will now be talking about two different
00:04:40;19 types of single point detectors, and then I'll switch
00:04:43;25 to cameras and explain to you more about how those work.
00:04:48;10 So the most classical single point detector is this
00:04:51;24 photomultiplier tube. So what that consists of is
00:04:56;15 a photocathode, and that photocathode is the element
00:05:02;08 that converts incoming photons into an electron.
00:05:07;12 So, once you have an electron, there are a couple of
00:05:12;17 so-called dynodes that attract these electrons and then
00:05:19;14 also accelerate them and multiply them. So at each dynode,
00:05:24;28 we get a multiplication of the number of electrons that
00:05:28;16 came in. We can tune the multiplication by tuning the voltage
00:05:34;10 over these dynodes. The net effect of all of this is that in the
00:05:39;19 end when we arrive here at the anode is that we have a
00:05:43;01 huge number of electrons for every electron that we get in here
00:05:48;09 initially. At that point, we're converting that charge into a
00:05:54;01 voltage, and then you read that out into a digital number
00:05:58;01 and send it to the computer. So PMTs are very fast.
00:06:03;01 They're highly linear. They have a very high gain, so
00:06:08;03 they can literally measure single electron hits.
00:06:11;15 So they're almost ideal measurement devices, with the main
00:06:18;18 pitfall is that they have a poor quantum efficiency. Quantum
00:06:21;24 efficiency means what is the efficiency which I convert
00:06:29;06 a single photon into an electron. So this quantum efficiency
00:06:33;28 of 25% means that you need, on average, 4 photons
00:06:38;03 to hit that photocathode to get one electron in, and
00:06:41;27 that electron you can then measure very, very well.
00:06:46;14 There are other types of designs for these PMTs, but
00:06:52;24 they all work on the same principle using dynodes. Here you see
00:06:57;13 a real example of what such a tube looks like. You can run them in
00:07:02;00 different modes. So one mode is the photon counting
00:07:06;17 mode, where you basically set a very, very high gain. So
00:07:10;19 that every photon that comes in and is converted into an electron
00:07:15;07 is being counted as a single pulse. And that way, you
00:07:20;29 count photons. The advantage of this is that you now have no
00:07:25;05 more background, because either you have a hit or
00:07:28;22 you don't have a hit. The downside is that this is very, very slow.
00:07:33;12 A very nice example of this photon counting mode is what
00:07:39;26 you see in Geiger counters. Geiger counters are basically
00:07:42;26 a PMT with a scintillation material in front of it so that
00:07:48;16 radioactive material is changed into a photon, that photon hits
00:07:53;26 the photon cathode. And with every hit is when you get a count
00:07:58;07 and you can hear that in your Geiger counter. So the other mode
00:08:03;18 is linear mode, where we measure the current in the
00:08:07;01 end. Which is a much faster mode of operation, but also
00:08:10;25 has more noise. Now in part, to overcome that low quantum
00:08:16;21 efficiency, there's a competing technology. And that is the
00:08:22;06 avalanche photodiode, the APD. This is based on semiconductor
00:08:28;25 material, it's actually very similar to the CCD sensors used
00:08:35;13 in cameras. So the semiconductor material absorbs photons
00:08:39;17 and then in this area here, there is again, a high voltage.
00:08:45;22 The electrons are accelerated, then hit that silicone with a
00:08:51;13 high force, and through a process that's called impact
00:08:55;01 ionization, they illicit extra electrons out of the material.
00:08:59;17 And that amplifies the signal, so it amplifies the charge,
00:09:06;01 the charge is being read out, and transferred in the end
00:09:08;29 into a digital number. These have a much higher quantum
00:09:13;02 efficiency than photo multipliers. They can also be used in this
00:09:18;04 photon counting mode. They're very useful. One of their
00:09:22;28 downsides seems to be that you cannot run them as fast
00:09:26;01 as photo multipliers, since they tend to overheat when you
00:09:30;19 go too fast. So this is an example of what an APD looks like
00:09:36;05 in reality. Okay, so that was it for single point detectors.
00:09:42;19 I will now be talking about cameras, and cameras in
00:09:46;02 microscopy tend to look like this, where there's a nice package
00:09:49;28 with the electronics and some cooling around that central
00:09:54;26 element that you see here, the actual photosensitive chip.
00:09:58;10 So, the chip is the heart of the camera. And as you may be
00:10:06;27 well aware, that chip consists of an array, rows and columns,
00:10:12;12 of the photosensitive material. That photosensitive material
00:10:17;09 itself is very much like that APD, consists of silicone
00:10:22;26 semiconductive material. They're produced in ways that are
00:10:29;10 very similar to how the chips in our computers are produced.
00:10:32;22 Photons hit that silicone through the photoelectric effect,
00:10:38;23 where photons are converted into electrons, and then they are
00:10:43;08 most often kept for a while in this potential well. So
00:10:47;25 charge builds up in this well here. It's the basic principle.
00:10:53;24 After that, we need to read that out and get those numbers
00:10:58;12 into our computer. And there are many different ways
00:11:02;00 and factors that influence that process. So first of all,
00:11:06;29 there are two main architectures for these photosensitive elements.
00:11:11;24 One is called CMOS and the other called CCD, for charged
00:11:18;05 coupled device. Probably both terms you've heard before. The main
00:11:23;02 difference is that a CMOS has itself a little analog charge
00:11:32;05 to voltage converter built into the chip itself. So there are
00:11:38;22 transistors here, right here on the photosensitive material,
00:11:42;21 that convert the charge into a voltage. That voltage is then sent
00:11:48;00 to an analog to digital converter and converted into a
00:11:51;27 digital number. A CCD on the other hand, does not have
00:11:56;05 this analog to voltage converter built in, that happens outside
00:12:01;09 at the end of the chip. And it means that all these photosensitive
00:12:05;24 elements share the exact same analog to voltage converter, which
00:12:11;07 in the end makes it easier to do that process exactly the same
00:12:17;00 for all photosensitive elements. And that is why, until not too long ago
00:12:23;20 CCDs were really the instrument of choice in scientific imaging.
00:12:28;05 It's no longer the case, and I'll explain that in the second lecture.
00:12:33;25 So in general, in a CMOS, each pixel has its own amplifier,
00:12:37;13 they're fast, but they tend to be much noisier. Although again,
00:12:43;19 there are newer variants that have lower noise and are
00:12:47;00 actually highly interesting. CCDs are slower, but they're
00:12:51;20 more precise, lower noise. Okay, so I'll be talking mainly about
00:12:57;20 how those CCDs work, how do they read out the charge
00:13:02;06 that they accumulate? And one beautiful analogy is this
00:13:07;20 bucket-brigade idea. So think about your photosensitive
00:13:14;00 wells as little buckets that are sitting on a conveyor belt.
00:13:18;26 It is raining, and we want to measure how much rain is
00:13:24;08 falling into each of these individual buckets. Well, what do we do?
00:13:28;23 We stop the rain, and then we start transporting this array
00:13:34;23 of buckets, one column at a time, into this serial bucket
00:13:41;26 array here at the end. So we get all the water from this last row
00:13:47;04 here into this serial row, and subsequently, we can now
00:13:51;03 read out bucket by bucket, into our measuring cylinder
00:13:55;22 how much water actually fell into each individual bucket.
00:14:00;15 And so as long as we keep our administration straight,
00:14:04;04 in the end, we know exactly which bucket had how much
00:14:07;18 water. You can for instance already see here or this analogy also
00:14:13;02 beautifully explains that if you want to read this out faster,
00:14:16;01 that you have to read out in this single element faster
00:14:21;15 and faster, how much water was in there. And you can see
00:14:24;12 that that's going to be less accurate. And that's in general
00:14:27;29 true. The faster you read out a CCD, the more noise you
00:14:31;20 get. Now, this is a very fair analogy. And it works extremely
00:14:38;24 well and you are perfectly fine to just remember this.
00:14:42;21 Nevertheless, I do want to give you a bit of an idea
00:14:45;27 of what is really going on. So, that CCD chip basically
00:14:52;07 consists of that semiconductor material. Here we're looking on
00:14:56;02 top. There are channel stops that keep the charge confined
00:15:02;00 between these rows here. And those electrodes are what's going to be
00:15:07;19 used to channel the charge through the chip. So here's
00:15:13;09 a cross-section, where we see the channel stops and
00:15:16;24 we see the electrodes on top, the semiconductor material
00:15:20;24 that will accumulate the charge. When we now start
00:15:24;21 an exposure, what is going to happen is that photons
00:15:29;13 will be coming in, they're being transformed into electrons, which we
00:15:35;05 often actually call photoelectrons. And as photoelectrons, they'll be
00:15:39;26 kept in this central part of the chip because we happen to
00:15:42;23 put a charge, a positive charge, onto that central electrode.
00:15:47;07 So this will keep the electrons right there under that
00:15:52;16 charged electron. So once we're done with that exposure,
00:15:56;28 we stop the incoming photons, we're now going to read it out.
00:16:01;19 And so, what is going to happen is that we're going to put
00:16:08;03 a sequence of voltages over these three electrodes on every chip
00:16:12;10 and that will in the end have a net effect that it will move the
00:16:15;26 charge through the chip. So up here, we see the voltages
00:16:22;10 that are going to be applied to these individual electrons.
00:16:26;02 So here at this point in time, we see that the voltage is
00:16:30;28 5V for this third electrode. And then when we now cycle
00:16:36;23 through, we now have both the second and third electrodes
00:16:41;26 high. That will smear out this electron package over 2/3
00:16:47;25 of the chip. Subsequently, we bring down that third electrode,
00:16:52;28 so we now already moved by 1/3 of the pixel that electron
00:16:57;01 charge. We do the same trick again with the next electrode
00:17:03;00 here. We bring that up, smear out the voltage package,
00:17:05;24 and so on, and so on. Until in the end, we move the charge
00:17:11;13 out of this pixel and into the next one. And so this process
00:17:15;18 repeats itself over and over and over again at a rate of
00:17:19;26 many million times a second to read out the charge that
00:17:25;04 was accumulated in your CCD chip. There's one little trick here, and
00:17:31;25 that's that I was telling you that it stopped raining at a certain point
00:17:36;12 in time. That the photons stop coming. In reality that's
00:17:41;22 of course not the case. So we must use some kind of
00:17:45;28 trick to stop that light from reaching the photo sensor.
00:17:50;05 And that problem is the basis for these different CCD
00:17:56;22 architectures. So the chip that I described was this full-frame
00:18:01;16 chip. And basically, in that case, what you have to do is when
00:18:06;23 you start reading it out, you have to put a shutter in front
00:18:10;07 of the chip so that no more light is reaching it. That's of course
00:18:16;03 not nice, because shutters are bulky, they make movements.
00:18:21;07 They are slow. So, there are a few other tricks. One trick
00:18:28;00 is to this so called frame transfer architecture. What happens here
00:18:32;22 is that the actual chip is twice the size of the part that you can
00:18:40;14 work with, where half the chip is dark, so there's like
00:18:44;20 an aluminum layer built on top of the chip so that no light
00:18:48;14 can ever reach it. You can now very quickly clock the charge
00:18:54;12 of this part of the chip in one go into the lower half of the chip.
00:18:58;17 And then you can start reading out that lower half, at leisure
00:19:03;06 or as fast as you want. You can already start exposing
00:19:06;22 the upper part of the frame transfer chip. This works very well,
00:19:11;07 except when you have very bright objects in that upper half,
00:19:15;10 and then you will see these lines smearing. So there's another
00:19:19;20 trick, and that is the so-called interline transfer architecture.
00:19:24;28 And so what happens there is we have alternating columns
00:19:28;21 of photosensitive cells and dark cells. And now we can in one
00:19:35;23 movement, we can go from the exposed into the dark
00:19:41;10 area. So we move from here to here in a very quick
00:19:46;10 move. And then we can read out that interspersed
00:19:50;00 dark area of the chip. And that has clearly a downside in that
00:19:54;00 like half the area of our chip cannot see light because
00:19:58;25 it's dark. And so one trick to get around that is to
00:20:02;24 put little microlenses on top of these two, and to direct
00:20:07;26 the light that would normally hit the dark pixel into
00:20:10;24 the light pixel. And so, these two architectures, frame
00:20:16;00 transfer and interline transfer are what will mainly be
00:20:19;18 seen later on. So, one remaining question, or one question
00:20:27;07 that is very often asked is, why are we not using color
00:20:32;27 cameras for this microscopy? And the reason for that
00:20:37;24 is that we really care about our light budget, so any
00:20:42;03 photon that is going to hit the camera, we want to
00:20:47;07 measure them. We want cameras with a high quantum
00:20:49;28 efficiency. We really want to see all the light that is possible.
00:20:53;28 And so a color CCD has the fundamental issue that that
00:20:59;23 doesn't help you in that regard, because the color CCD is
00:21:04;04 nothing else but a monochrome CCDs with little colored
00:21:08;06 masks on top of each pixel. And the most commonly mask
00:21:12;27 is called a Bayer mask, and consists of two green pixels,
00:21:16;12 a blue, and a red one. The images being read out.
00:21:22;20 The intensity information is being taken from each
00:21:26;21 pixel, but the color information is actually calculated in the
00:21:31;19 computer or in the firmware of the camera from the
00:21:34;23 information of those neighboring differently colored pixels.
00:21:38;19 And so that means that we throw away at least 2/3 of
00:21:44;02 the light at every pixel. And we also get like an interpolated
00:21:49;29 image, and not really the real image. So, in most, by far most
00:21:55;10 fluorescence microscopy images, where we care about the
00:21:59;08 photons, you will not see color CCDs. So that's it for
00:22:06;19 this first part of detectors. I do want to mention Kurt. Kurt Thorn
00:22:12;08 who has contributed quite a bit of material to this talk.
00:22:16;07 Also, this website micro.magnet.fsu.edu is a great resource
00:22:23;10 for everything microscopy related, and there were a few
00:22:26;28 illustrations that I used from them. And I also used some
00:22:31;20 materials from Wikipedia.

This Talk

Speaker: Nico Stuurman

Audience:

Researcher

Recorded: April 2012

Talk Overview

Photosensitive detectors are used in microscopy to generate digital images. Nico Stuurman introduces the two main classes of photosensitive detectors: single-point (Photo-multipliers and Photo-diodes) and multi-point detectors (cameras) and explains their basic principles.

Questions

Consider the following detectors:
1. PMT
2. APD
3. CCD
4. CMOS
  i. Which two of the above are multi-point detectors?
  ii. Which of the above are single-point detectors?
What is the main advantage of an APD over a PMT?
1. Higher gain
2. Faster response
3. Higher quantum efficiency
4. Greater linearity
Which of the following statements concerning CCDs and CMOS architectures is true?
1. Both store photo-electrons in a potential well
2. Both contain transistors in their photo-sensitive elements
3. Both contain only one (or a few) read-out amplifier
4. CCDs are usually read out faster than CMOSs.
In the CCD interline transfer design,
1. The image is transferred to a dark storage area where it is read out
2. The image is read out normally, but smearing does not occur because of a shutter in front of the sensor
3. The whole image is shifted completely into the serial register from which it is read out
4. In a single shift, the image is shifted into dark pixels, interspersed amongst the photo-sensing pixels

Answers

Speaker Bio

Nico Stuurman

Nico Stuurman

Nico Stuurman is a Research Specialist at the University of California, San Francisco, in the lab of Ron Vale. Nico combines his expertise in computer programming and microscopy to advance many projects including the Open Source software, Micro-Manager. Continue Reading

Sign up for the Science Communication Lab education newsletter