Vector Vs. Raster: The Fool's Game

Craig Hockenberry recently posted an article at the IconFactory regarding resolution independence in image processing. This is part of an ongoing public debate regarding the merits of using vector graphics instead of raster graphics for user interface elements, especially icons. I'm too lazy to link to all of the other people involved in this debate; there's a lot of them. Here are two of the salvos:

To paraphrase the problem for those just joining the party: monitors are getting bigger, pixels are getting smaller, and icons are becoming increasingly-larger in order to maintain their relative size on-screen. Some people think that vector-based images are the way to go, because their files are smaller and can scale cleanly to any size. Some think raster-based (pixel-based) images are the way to go, because even if they become very large, they can be specified exactly and are able to be drawn quickly.

This debate is becoming less academic and more pragmatic as time goes on, because Mac OS 10.5 Leopard supports icons that are up to 512x512 pixels large. Which, yes, is crazy.

I'd like to address Hockenberry in particular, because some things in his article really stood out for me. Hockenberry's article has the following main points:

  • Today's icons require lots of vectors and are consequently huge
  • Rendering vector icons is slow
  • Vector icons look bad when scaled to small sizes

Someone's Snorting Something

To support his first argument, "vectorized icons are huge", Hockenberry posts two files -- a 512x512 rasterized PNG image of the CandyBar icon, and a corresponding PDF file that contains the same icon in vector format. As the PNG image is 100KB, and the PDF is about 30 times that size, he draws the conclusion that vectorized icons require more space.

Unfortunately, on this point he is dead wrong. If one uncompresses the PDF file (using pdftk, for example), you can examine the raw data inside of it. What does it contain? A total of 281 rasterized images, presumably one for each layer of the image:

 sdeken01@alondra: ~$ grep -i --binary-files=text "/subtype /image" CandyBar2.pdf | wc -l
 281

How much space do these images take? We can fortunately rely on the output to have a /length tag two lines following the /subtype /image tag, so we just need to look for them and tally them up:

 sdeken01@alondra: ~$ grep -A3 -i --binary-files=text "/subtype /image" CandyBar2.pdf | grep Length | awk '{ print $2 }' | perl -wlne '$t += $_;}print $t;{'
 48911164

Wait, how big was this file again?

 sdeken01@alondra: ~$ ls -al CandyBar2.pdf
 -rw-r--r--  1 sdeken01 pg528391 49306138 2006-11-15 08:05 CandyBar2.pdf

So 48,911,164 bytes out of 49,306,138 -- 99.2% -- are consumed by these images. That means that the actual vector data consumes roughly 394,974 bytes. That's still almost four times the size of the 103,102 byte PNG file, but it's not so bad.

We Don't Need No Steenking Compression

Of course, this is all assuming that PDF is the best format to use to store your vectorized icons. I submit that it is not. PDF is a verbose format which is geared toward representing anything that can be in a document -- text, vectors, tables, footnotes, etc. PDF exists to preserve the formatting of a document that is intended to be printed to a printing device or a computer monitor. It is far too heavyweight for the purposes of representing a vectorized icon.

In the long term, a better approach would be to develop a specialized vector icon format. (No, SVG isn't it. It's worse than PDF in verbosity.) A single Bezier path control point fits into exactly 24 bytes (48 if you want crazy precision), so in a 100KB file you can fit 4000 or so path nodes without any sort of compression. I have no easy way of counting the number of path nodes in the CandyBar vectorized file, but 4000 is an awful lot of points.

If we add compression, and we assume that we can get half the compression of the PNG file (which compresses the 512x512 image with 4 bytes per pixel from an original size of 1.05MB to 103KB, or 90%), we should be able to store close to 10,000 Bezier curve points in a 100KB file.

The gradients don't matter. They should add close to 24 bytes each, the same as a normal control point (8 bytes for the XY coords of each end of the fill, and 16 bytes for the start and end colors).

Rendering Vector Icons Ain't Slow

It's true that rendering the 8MB monstrosity that is the CandyBar PDF is an excruciatingly slow process. But this doesn't mean that rasterizing a vector image is slow. Computers do this all the time. Near-constantly, in fact. I suspect that just before you read this, your computer rasterized every letter in this essay using a vector-based font.

Did you notice a delay?

It's certainly true that rendering a vector-based image takes more time than rendering a raster-based image. However, the difference is small, and it can be mitigated in a number of ways. One way is to use intelligent caching, so that the image is rendered once and then resampled to produce smaller versions. Another is to leverage the graphics card to do some of the rasterization for the CPU.

But in general, rendering a vector image is not the slow, excruciating process that Hockenberry's PDF makes it seem to be.

Let Me Give You A Hint

The final argument -- that rendering vector icons at a small size produces worse results than placing pixels individually -- does hold some water. As the icon gets more cramped, there are more details competing for less pixels, and it's certainly true that an intelligently-designed 16x16 icon will beat a 512x512 icon scaled down in terms of crispness and general look.

But.

A generalization of this argument was put forth earlier by Josh Williams of Firewheel Design, and it's always sat the wrong way with me. Williams sets forth an argument that an icon designed for a specific resolution will look poor when scaled either up or down, because detailed line work which is intended to produce a single pixel at a particular resolution will be mapped to either multiple pixels (when scaled up) or fractional pixels (when scaled down).

Williams' argument bothers me because this is another problem which has been previously solved in the world of typography. Most major fonts have been extensively hinted so that they are legible at small point sizes. Verdana at 12 pt may seem like a bitmap on the screen, since it only stands 10 to 12 pixels high and does not contain any shades of grey, but it is stored on disk as a vector. The same font that produces the highly aliased 12 pt version on the screen is the same font that produces the highly anti-aliased 72 pt version on the printer.

It seems reasonable to me that a mechanism similar to hinting could be created for icon design. If you have a design element which is intended to be exactly one pixel wide, you could add hints to your control points to ensure that it uses exactly one pixel at every resolution. Or, as your design gets scaled down, you could use hints to remove paths that would result in unnecessary clutter, allowing the major design elements to stand out.

We do not have the tools to do such things today, and I'm not sure we ever will. After all, storing a customized 16x16 raster image doesn't take much space, and it's less cumbersome to work with than a series of carefully-crafted control point hints. I would expect icons to move toward a hybrid vector-raster format in the future, where certain sizes are represented with a raster image, and larger sizes are rendered from vector images. Apple has already laid the groundwork for this with their ICNS format, and some open source groups (such as the Tango project) are starting to get the same idea.


Hat-tip to John Gruber for the link. John notes:

Arguing for a new vector format optimized for iconography is not the same thing as arguing that vector art should be used for producing higher-resolution icons today.

I don't think anyone really disagrees about using vector art to design high-resolution icons. The majority of the 'sexy' icons on the Mac today are rasterized versions of vector icons -- the Firefox icon is vector-based, all of Jasper Hauser's icons are vector based, and there are plenty of other designers who use vector art extensively when producing icons.

The only question is: should these vector forms be distributed and used to render the icons directly on the fly, or should they be pre-rasterized and touched up for certain dimensions? I submit that we should have our cake and eat it too.

Update update: John meant distributing, not producing. Case closed!

24 comments on ‘Vector Vs. Raster: The Fool's Game’

I'd point out that SVG is not designed to be terse - if it were, they wouldn't have started from XML.

The big point about SVG is that it compresses *really* well, especially in combination with, say, gzip - and every SVG viewer/editor I'm aware of knows how to handle .svgz files, which *are* very compact.

by clee
Wednesday, November 15, 2006; 12:26 pm
True enough -- compression with gzip certainly takes a majority of the size away, usually by about 50% or more, and comes pretty healthily close to the size of the binary format I described.
Wednesday, November 15, 2006; 12:34 pm
On a high level, this indeed makese sense, but there are instances where vector in any format would surely make a larger file. For instance a pattern of small circles/grid (like in the OSX trash can). The number of points describing each "hole" would far exceed the number of pixels they represent. How would this be solved through a purely vector rendering?
Wednesday, November 15, 2006; 1:12 pm
In theory, the idea of being able to hint a vector icon to perform differently at different sizes would be nice. However, the geometry of given letter in a font is much less complex than the geometry of a vector icon.

In effect you're still creating altered states of the same vector icon, and you could argue that you could spend your time either hinting the vectors or creating custom bitmap resources.

The reality is a one-size-fits-all vector image for typical usage within an application is a pipe dream.

In regards to your suggestion that you could get 10,000 points in to 100k file... well, I'll be a believer when I see it.

Wednesday, November 15, 2006; 1:33 pm
Stephen, you make the case that "rendering vector icons ain't slow." You point out that one can stuff a lot of vector nodes into the same space as a 512x512 raster file.

Let's chat about the CandyBar icon. How many vector nodes do you think it would take to describe it accurately? And how much more processor cycles would it take to render all those nodes, compared to the bitmap?

I know you (or I) can't answer those questions precisely, but would you agree that it would be a *lot* of vectors, and that the difference in processor cycles required would be at least an order of magnitude, even if the bitmap were scaled on the fly?

by Splashman
Wednesday, November 15, 2006; 2:05 pm
I use SVG in an even more compact way then svgz. Try my Adobe plug-in example sites or if you have a copy of the nightly build of Webkit try this link;

http://deerring.com/gfonts.svg

Enjoy!

Wednesday, November 15, 2006; 3:10 pm
Core Image & Core Animation show that vector art (and filters) can be very fast and very exciting. Imagine an icon format based on Core Animation! All pulsating and shiny and blurry in cool places.

This is where Apple should be heading :-)

Wednesday, November 15, 2006; 3:59 pm
Have you heard of the new icon format for Haiku (an open-source BeOS clone in progress)? It's a vector format designed specifically for icons.

General introduction: http://haiku-os.org/node/280

How it works: http://haiku-os.org/node/282

Wednesday, November 15, 2006; 4:02 pm
This example of typography that you cite is more interesting than you suspect, I think. After all, one of the more persistent complaints of type nuts is that the one-size-fits-all type scaling has destroyed a lot of the integrity of fine typography.

I'm not sure I buy it, but the default scale of font sizes that computers use was standardized exactly because fonts shouldn't be exactly the same at different sizes. It's often more than just minor differences, too: Fonts designed specially for print often have a display weight for large uses, with thinner lines, larger ascenders and descenders, and a smaller X-height. And extremely specialized fonts (for instance, fonts for newspaper body or agate copy) will be designed for use only between, say, 8-10 points, or 5-7 points.

It may be that designing one icon for all possible scaled sizes is as insensitive to the design as one letter shape for all optical sizes.

Wednesday, November 15, 2006; 8:43 pm
You said: "[H]e draws the conclusion that vectorized icons require more space. Unfortunately, on this point he is dead wrong."

Then you said: "That means that the actual vector data consumes roughly 394,974 bytes. That's still almost four times the size of the 103,102 byte PNG file"

Four times the size of the PNG = requires more space = not dead wrong

by
Wednesday, November 15, 2006; 8:57 pm
I loaded the PDF of the Candybar icon and immediately noticed both how slow it drew, and how inefficiently it was constructed. There were massive gradients that were almost completely obscured by objects on higher layers, like the the drop shadow. These areas could easily have been created more simply, omitting areas that were not visible in the final rendered image, which would have required much less memory. So the file size could have been even smaller than your analysis indicates.

sambeaus observation about core image icons is interesting. We already have similar icons, you can put a playing QuickTime movie in the dock and the icon plays continuously. There is much untapped goodness in the current icon system.

Wednesday, November 15, 2006; 11:34 pm
Hm. Vector image format, more compact than SVG?

Sounds like Encapsulated PostScript to me.

The only downside to EPS is that on Mac OS X, EPS support is implemented by converting to PDF, which is a *very* slow process (I don't know if this is by nature or if it simply has not received much optimization attention).

I'd like to see Apple add native support for EPS, as well as SVG and maybe even HVIF, to the ImageIO framework. And I'd like to see them extend Icon Services to support obtaining any icon size from such a file.

Thursday, November 16, 2006; 12:25 am
Peter, the other downside is that EPS does not support transparency information. True transparent drop shadows and blurs are not possible with EPS.
Thursday, November 16, 2006; 3:17 am
The Haiku project just adressed this by making their own custom vector format: haiku-os.org You might want to look at this, the sizes are incredibly small around 1KB or less. Max icon size 64x64
by nutela
Thursday, November 16, 2006; 4:04 am
I don't see the point in caring about the small size icons. Even if you hand-position every pixel, as displays sizes grow and dot-pitches shrink, the detail will be lost on the user all the same.
Thursday, November 16, 2006; 7:26 am
The Haiku icons are a great, but good luck convincing Apple to convert all their icons to a simplified style. And I doubt you'll find any of those icons on any product packaging either. If you are developing for a major OS, then you pretty much have to have a complex icon.
by Tom Houser
Thursday, November 16, 2006; 7:41 am
Great comments, all.

Jordan, you're absolutely right that for very detailed work vectors can result in larger files. Each point takes up the equivalent of 8 rasterized pixels, so if the image is sufficiently small, the vector art would consume more than the rasterized art.

Josh, it's not difficult to produce a compressed SVG with 10,000 points that consumes around 100KB, and that's still bigger than what a specialized binary format would provide.

Splashman, the CandyBar icon isn't that much more complicated than the Firefox icon. The Firefox icon contains just under 500 points, plus perhaps 120 more for gradient targets. I'd say that 1000 points is probably a reasonable upper limit to the number of points that would be stored in an icon.

Rendering this amount of data really doesn't take very long, and as I said, the resulting raster image could be cached to make future drawing faster.

James, I couldn't get your SVG example to work -- I had problems getting my WebKit nightly to load its nightly version of the framework. Looking at the source, I see what you're doing -- a customized font that contains the artwork for each shape. Interesting approach! I do note that the font had to be encoded to fit into ASCII though, which expands its size.

Wes, I totally agree that a 'one size fits all' approach is foolish. It would take much longer to massage each point of an icon for a smaller image than it would take to manually retouch the raster version, and the result would still be inferior to a rasterized version. A hybrid approach would probably be best, with sizes above a certain point being rendered from the vectors.

Charles, it probably isn't worth it to cut out the parts that can't be seen -- it might actually add to the total number of points, and it might cause display issues when dealing with partial transparency. Plus, having all of the layers in there allows for more manipulation, either by an artist or by a developer -- imagine if the trial version of CandyBar slowly wrapped or unwrapped its icon to indicate how much time you had left in the trial!

Ultimately, any format is going to have positive and negative aspects to it, either with size, speed of rendering, quality, or ease of use. It all comes down to finding a balance of features that allows artists, developers, and users to all get things done.

The haiku icon format looks interesting; is it really limited to a maximum icon size of 64x64? That seems foolish to me in these recent days.

Thursday, November 16, 2006; 7:50 am
First, let me begin by saying that I've enjoyed reading all of these comments, even if some are a bit inflamatory -- I'm always open to constructive criticism :-)

I'd like to address all of the people who are saying, "Let's use something besides PDF!"

In the world I live in, that's not an option -- PDF is the only vector format supported by Mac OS X. Arguments in favor of SVG or specialized icon file formats are pie-in-the-sky wishes -- while you're at it, please wish for 3D holographic displays and 2400 DPI ePaper (I'd love a Mac with those technologies.)

I feel like I should address some of the comments about speed, as well.

I agree that rasterizing a vector is fast -- as long as it's simple. It's true that font outlines controlled by Bezier paths display quickly. But you must also realize that fonts don't have multiple colors, gradients, shadows, layers or any other kind of complexity. Comparing fonts to illustrations isn't fair.

Although I'm no expert with regard to the Mac OS X graphics rendering pipeline, all the information I've seen indicates that it's based on bitmaps, not vectors. There's a reason why you have Core Image and not Core Vector. If you think about it for more than 30 seconds, you'll realize that there would be a lot of branching if vectors made it into that pipeline (imagine a backing store containing a mixture of bitmaps and vectors and you'll start to cry.)

That's also the reason why the OS renders PDF graphics into bitmaps at the size it needs when you specify it as a part of your UI. Of course this bitmap is cached so you're not paying a penalty for its use, the hit will occur whenever it has to be sized to the display DPI. But don't count on that happening infrequently -- changing your display DPI may be as easy as holding down the control key and using your scroll wheel (similar to what's in Tiger today.)

As far as the size of vector icons is concerned, I'll be posting a follow-up tomorrow (11/17/2006).

-ch

Thursday, November 16, 2006; 11:07 am
IRIX has vector-icons. Seems to work flawlessly even on "old" hardware, by todays standards.

sgi have been there, done it and got the t-shirt, so it has been possible using vector-based icons for years on IRIX, why should it suddenly be not workable??

Thursday, November 16, 2006; 9:08 pm
Craig, two things:

(1) But you can't use PDF for icons on OS X today, not application (Finder) icons. It is possible to design an efficient vector format with gaussian blur, et cetra. It is also possible for Adobe to add them to PDF. The CandyBar icon should not be that huge in vector form. You can certainly represent complex gradients as vectors. Adobe should add the support you need to the format if it isn't there today (I know that simple gradients are available in PDF and supported by the PDF rendering engine in OS X, don't know how complex the support currently might be).

(2) The rendering pipeline -- sort of. OpenGL is entirely based on vectors, and it underlies the window server. Window backing stores in OS X are usually bitmaps today; they're sent as textures mapped on top of vectors (to simplify the terminology). But when you enable Quartz 2D Extreme/QuartzGL, this changes. Data (line drawing, etc) is actually sent to the GPU as vector data.

by Chris
Friday, November 17, 2006; 7:33 pm
Steve, the Haiku format doesn't even support for alpha. It is a colossal waste of time on somebody's part.
by Chris
Friday, November 17, 2006; 7:34 pm
"Haiku format doesn't even support for alpha"

That is not correct. It does actually support alpha.

The implementation of the icon format in haiku is actually very clever. A lot of emphasis has been placed on making it small and fast. I'm interested in seeing it running and looking at how the icons actually render.

Saturday, November 18, 2006; 4:48 am
"You can certainly represent complex gradients as vectors"

Simply not true. There's so many things that vectors can't do. I'm not suggesting I'm an Illustrator guru, but I have been using it professionally since version 3. Some things are just not possible, or so incredibly complex that they're not easily achievable (plus, the results are a technical nightmare).

I don't think some of you realise the other issues associated with vector artwork when used like this. There's so many issues and so little benefit. I really don't see the need.

I've posted some other comments here:

http://islayer.com/blog/?p=87

Sunday, November 19, 2006; 6:33 am
What are you talking about? Is this a code for cursing or something I dont know about???????
by Mom
Tuesday, November 21, 2006; 9:46 pm

Add a comment to ‘Vector Vs. Raster: The Fool's Game’.