« Wizards Win! | Main | Populism, Pro and Con »

Smart Folders

It seems that I initially underestimated Apple's new Tiger operating system. Spotlight isn't just a cool new search feature. The same technology lets you create something called Smart Folders -- basically a folder that, instead of containing some fixed set of content is pegged to a series of search criteria and "contains" whatever on your hard drive fits the bill. At first glance, that sort of seems like nothing more than a neat parlor trick, but having played around with it a little while it's clear that this is a huge deal. It will probably need to wait for the emergence of a new generation of computer users who aren't loaded down with legacy habits, but it's a method of data organization that I think could transcend the entire "files inside folders inside folders" paradigm that's dominated computer usage since Mac OS first came out and then Microsoft mimicked it.

May 7, 2005 | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Smart Folders:


oh, that's ingenious. I'm going to get Tiger now.

Posted by: praktike | May 7, 2005 2:09:59 PM

""files inside folders inside folders" paradigm that's dominated computer usage since Mac OS first came out and then Microsoft mimicked it."

This paradigm long predates Mac OS. Both Unix and VAX/VMS had this structure (with folders called "directories"), as did DOS and as I recall, CP/M. As far as directory structure, the Mac simply reified that structure in their GUI.

Posted by: EKR | May 7, 2005 2:11:00 PM

the emergence of a new generation of computer users who aren't loaded down with legacy habits

Oh, those pesky legacy users.

Me, I'm just lucky I escaped punch cards by a few years. ...and was successfully rehabilated from using "goto".

So which do you think is better, green or amber?

Posted by: PaulC | May 7, 2005 2:21:41 PM

As someone who's deeply using OS X lately (job), I have worries about Spotlight. It seems like it's going to put a lot of wear and tear on the file directory system that the MacFS uses, and seems to be a weak point of the operating system at large.

I hope it doesn't result in weird permissions errors/disappearing files/moving files, etc. that the MacFS is prone to do.

Posted by: Karmakin | May 7, 2005 2:21:56 PM

Let me restate that. I hope that doesn't result in MORE of that weirdness that I mentioned.

Posted by: Karmakin | May 7, 2005 2:22:38 PM

I actually don't know how Spotlight works, Karmakin, but it probably doesn't actually move any files into Smart Folders, because it would result in too many duplicate files in folders with similar criteria. Smart Folders probably just hold pointers to files stored in some original location.

Posted by: Kiril | May 7, 2005 2:31:01 PM

sounds like basically a relative database app to me, the "smart folder" being a simple query. If it is, it would require the user to label/name/tag information in a consistent way to be of any real use.

Posted by: hyh | May 7, 2005 2:40:31 PM

Hyh, that's sort of right - it's a live search query that returns a bunch of file aliases pretending to be a folder. But the query can use any of the criteria that are available for file searching, including file content and a user-defined field called "Spotlight comments". So a smart folder could be, say, all MS Word documents modified within the last month that contain the word "asparagus".

What makes this work with any reasonable speed is that files get reindexed in the background whenever you modify them.

Posted by: EliB | May 7, 2005 2:47:19 PM

sounds like basically a relative database app to me, the "smart folder" being a simple query.

Wouldn't it be more analogous to a view? I.e., a table defined by a query.

Posted by: PaulC | May 7, 2005 3:01:45 PM

How Spotlight works. Fairly technical.

Posted by: cwk | May 7, 2005 3:05:50 PM

sounds like basically a relative database app to me, the "smart folder" being a simple query.

Yes, except it's all happening at the file-system/kernel level for spotlight, which makes it much better.

Posted by: Kieran | May 7, 2005 3:06:41 PM

For instance, see this little demonstration --- scroll down / search for "boogawooga" (no, really).

Posted by: Kieran | May 7, 2005 3:08:51 PM

Ignorance Alert!

The concept of files in hierarchical "folders" or "directories" long predates the Macintosh.

Posted by: VividYou | May 7, 2005 3:19:05 PM

If this sort of thing floats your boat, you might want to read up on the concept of folksonomy. And take a look at You're It, a group blog devoted to examination of tagging and folksonomy.

Posted by: Avram | May 7, 2005 3:26:43 PM

Agreed with the directories are folders are files comment. Really old systems didn't have directories and didn't neccessary think of stored data as files either. And this won't eliminate that, it will depend on it.

I'd be curious how the internal structure works. If it's simply a collection of single-search-to-a-database files stored off in an individual directory somewhere, I would have serious questions about its robustness. On the other hand, if they're stored in separate directories with distinct symbolic links and the actual query stored as file (I am assuming it is storing individual document indexes with the indexed file) that would probably be pretty robust. It would certainly be very Unix-y.

['Do dah...']

Posted by: ash | May 7, 2005 3:28:42 PM

My copy of Tiger came yesterday afternoon, but alas I was or campus at an awards ceremony, so I didn't get it. Stupid award! ;-p

I am ready to try it out. Unlike most folks, though, my reasons are very geeky. The fancy UI stuff will be neat, but I am really looking forward to the (finally) complete 64 bit integer support. There are several numerical algorithms I have that I can do fun things with because of that (it's a boon for cryptology). Also, long long is implemented in the new compiler, so I can do some diff. eq. numerics with 128 bit precision, too.

Posted by: teece | May 7, 2005 3:37:41 PM

Mozilla Thunderbird does a similar thing when you search for email messages. You search via a any number of criteria then when you have a list of results you can create a "folder" that holds the contents of that search.

Posted by: kid g | May 7, 2005 3:41:46 PM

Interesting. Sounds like Apple came up with a simple way to mimick aspect's of WinFS.


Interesting. So Microsoft pre-announces... Apple mimmicks, and then is given credit for "innovating".

Sort of like how they invented the whole idea of files and folders and everybody else mimmicked it back in the 1970s. :-)

Posted by: J. Caesar | May 7, 2005 3:52:13 PM

Spotlight is indeed the Big Deal about Tiger.

And once you start to understand the implications, it does promise some revolutionary things. Microsoft is working on an implementation of somewhat similar ideas for Longhorn, although I'd bet they're going to miss some of the heart of the matter.

One nice thing Apple did with Spotlight was to give it plenty of hooks for developers to build some really interesting apps using system wide metadata. The supplied Tiger UI for Spotlight is pretty primitive - much more is possible.

And if you're looking for history, Spotlight is essentially a watered down version of the metadata system in the now deceased BeOS

Posted by: Petey | May 7, 2005 4:04:56 PM

Apple has a nice page o' tips for the wanna be Spotlight power user.

Posted by: Petey | May 7, 2005 4:06:20 PM

Maybe I missed something, but Matt's comment doesn't say that Apple invented folders and files. It does say that folder/file organization became *dominant* after the Mac OS came out, but that's not the same thing.

What's more, Matt's right. Folder/file organization seems tied to a graphic user interface to me. DOS/Unix directory structures are logically similar to the folder/file organization, but because OSes like Mac/Windows/etc. let you manipulate files/folders as visual objects, the user experience of the filesystem is very different. Even if Apple didn't invent the GUI (it was Xerox labs, right?), there's no question that the Mac played a crucial role in popularizing the GUI.

Posted by: Chris Lovell | May 7, 2005 5:51:09 PM

I use Adobe Photoshop Album which allows one to create catagories and subcatagories of photos stored anyhere on the host computer. This sounds like the same concept as smart folders.

It is implementsd in Album by attaching tags to photos that identify the catagory they are assigned to and I suspect a data base query is performed when a catagory is opened. Looks just like opening a folder on the computer but is slower due to the query.

Posted by: Robert Brown | May 7, 2005 6:22:12 PM

Smart Folders are a continuation of the app-centric 'submerge the hierarchy' thing that Apple has done with iTunes and iPhoto. It's clear that what worked on 1Gb drives doesn't really scale: people have a lot of stuff on their hard drives these days, and that stuff doesn't necessarily file hierarchically. It fits a model that's closer to SQL searches than files in boxes in boxes in boxes.

Interesting. Sounds like Apple came up with a simple way to mimick aspect's of WinFS.

Nah, WinFS is 'inspired' by BFS.

Any file or file type on a BFS volume can have arrays of metadata associated with it, in the form of "attributes." There is no limit to the amount, size, or type of attributes, and attributes can be displayed and edited, sifted, sorted, and queried for directly in the Tracker (Be's equivalent of the Finder). Because most attributes are indexed, search results are nearly instantaneous, regardless the size of the volume or the number of files being searched through. By default, BeOS ships with reasonable sets of attributes for common file types, but users are allowed to extend and customize these, and to create entirely new file types with entirely new arrays of attributes. In other words, the Be File System doubles as a database.

Apple hired the BeOS file system guy to work on OS X. Most people who used BeOS lament the loss of the BFS most of all, because of its metadata support. Smart Folders combined with Spotlight file metadata looks like it's going to be really sexy indeed.

Posted by: ahem | May 7, 2005 6:33:49 PM

The desktop metaphor is a graphical fix on top of directory hierarchies, which were popularized by the various unix flavors (if memory serves, Dennis Ritchie wrote a couple of papers on the idea of a directory-based file structure). Apple's major achievement was to popularize the desktop metaphor itself, even if the metaphor breaks down with things that make sense in a hierarchical directory structure (such as aliasing) that don't make as much sense in a desktop. As far as CS metaphors go, it's not that bad (the use of "key" in cryptography is still the thing that drives me the closest to murder), but its fairly easy to watch it fall apart and be starting at inodes in all of their filthy glory.

What's interesting to me about spotlight and smart folders is how it ties in with some ideas I had the chance to play around with a few years ago - flat information spaces which are basically sculpted by search functions. If you look at most of our web tools - link lists, blogs, and so on, they are really ways of taking a flat space (since int he web, everything is equidistant) and applying some idea of proximity and distance to it. I hacked up some small apps a few years ago to play with the idea (warning: java), and there were some companies that fumbled towards the idea in an interesting fashion before dying with many other dotcoms. I gotta get back to that someday.

Posted by: Mike Collins | May 7, 2005 6:43:25 PM

Even if Apple didn't invent the GUI (it was Xerox labs, right?). . .

The developers of the Xerox Alto and Star are credited with inventing the most common metaphors in GUIs today, based on the desktop, but graphical user interfaces go back to Ivan Sutherland's work in 1963 on Sketchpad, which was the first full-up GUI, with non-batch input, a pointing device for direct manipulation (a light pointer, but Sutherland later invented the mouse), a graphical display, and so forth. There are videos of Sketchpad online, and they're pretty impressive. Alan Kay (another important figure in the history of interactive computing--see his Squeak system for a somewhat different take on GUIs today) mentioned in a talk a few years ago that today's GUIs still haven't caught up to Sutherland's vision.

Posted by: RSA | May 7, 2005 7:43:06 PM

The comments to this entry are closed.