Does the Mac filesystem need defragmeting? [Archive]

Marcwic

03-19-2002, 12:56 AM

Thanks for the reply. One of the things that would be cool would be if you could have other file systems on X like SGI's XFS (i think thats what it's called) which doesn't need defragmenting or error checking if you don't shutdown prop

Marc

Originally posted by griffman
I'm not aware of any free defragmenters. Micromat and Norton both make defrag products that will work with OS X drives, since they're just HFS+ (but double-check their websites to be sure).

In 15+ years of using my Mac, including 1.5 with OS X, I've yet to defrag a drive. If I had less RAM, maybe I'd notice a problem, but my machine certainly seems just as peppy as it's ever been.

-rob.

djidji

03-20-2002, 06:29 AM

Mmm, I think you should check again. Before I swithed to MacOS X I used to defrag on a regular basis, using Norton. Saves a lot of trouble. Now the OS is different, but the fs is the same. I think we do need some defragging soft for MacOS X

Marcwic

03-20-2002, 07:04 AM

Originally posted by djidji
Mmm, I think you should check again. Before I swithed to MacOS X I used to defrag on a regular basis, using Norton. Saves a lot of trouble. Now the OS is different, but the fs is the same. I think we do need some defragging soft for MacOS X

Hopefully it will be free and won't require a restarting, although a commandline based on that runs in single user mode would be a good option to make sure the disk isn't be used much.

djidji

03-20-2002, 07:13 AM

heeey!
what about iPod? it's got an HFS (or HFS+) partition, how about a defragmenting tool for it! thats an idea for commercial soft, imagine "structures all your playlists for better playback" on the sticker

Marcwic

03-20-2002, 07:30 AM

Originally posted by djidji
heeey!
what about iPod? it's got an HFS (or HFS+) partition, how about a defragmenting tool for it! thats an idea for commercial soft, imagine "structures all your playlists for better playback" on the sticker

LOL! They could charge a packet for it as well.

brodie

03-20-2002, 05:10 PM

i personally think you spend more time opening the application and clicking yes/no to all the "shall i fix it?" than you would actually save in the long run, and the unmounting/cpu usage also costs you.

typoon

04-09-2002, 04:10 PM

Try going to Versiontracker. There is a program called XOptimize that works pretty well and it's free. You should still run Norton and Diskwarrior to optimize your drive to see any real speed in OS X

Marcwic

04-09-2002, 04:52 PM

thanks for that, XOptimize looks good.

griffman

04-09-2002, 05:23 PM

Just to clarify, XOptimize is in no way a disk defragger. It's merely a front end to the Terminal command:

sudo update prebinding -root /

This command simply helps OS X launch applications faster. It doesn't physically move anything around on your hard drive. It simply pre-binds (hence the name) the libraries that the application needs to load at runtime, so that when launched, the application doesn't have to go looking for them.

This is a much different beast than a disk defragmetation program, and it is something that I do regularly.

-rob.

Marcwic

04-09-2002, 05:25 PM

Do you have to do it each time you start your mac, or just once?

Originally posted by griffman
Just to clarify, XOptimize is in no way a disk defragger. It's merely a front end to the Terminal command:

sudo update prebinding -root /

This command simply helps OS X launch applications faster. It doesn't physically move anything around on your hard drive. It simply pre-binds (hence the name) the libraries that the application needs to load at runtime, so that when launched, the application doesn't have to go looking for them.

This is a much different beast than a disk defragmetation program, and it is something that I do regularly.

-rob.

xchanyazy

04-09-2002, 05:28 PM

You never have to update the prebinding - it's done automatically by many installers (the Optimizing System Performance step), and applications won't fail if it's not done. Just a little bit of preventative maintenance, it's good to do after you install a bunch of programs just to optimize a little.

Cheerios

04-11-2002, 03:33 AM

so what about defragging, then? I was told that UNIX does all that upkeep on it's own, while the system is running, which is why there's no defrag app in the system utilities, but I've never had it confirmed...

Marcwic

04-11-2002, 04:01 AM

Originally posted by Cheerios
so what about defragging, then? I was told that UNIX does all that upkeep on it's own, while the system is running, which is why there's no defrag app in the system utilities, but I've never had it confirmed...

Defragmenting is putting all the data on a disk together, over time data gets scatterd, so it takes longer to applications to start, and for large documents to open.

Defragmenting moves the data into one place on a disk, Unix doesn't do this unless you have a filesystem that can, the Mac OS filesystem can't.

Windows includes a defragmentor, see http://members.aol.com/marcwic/defrag.jpg

GerryA

04-18-2002, 07:47 AM

I read somewhere that the unix answer to defragmentation is to ensure as much as possible that the fragments are on the same sector or cylinder or whatever to ensure that the drive doesn't have to zip all over the surface of the disk to find the different fragments. Ok, so you can tell that I don't really know what I'm talking about, but it sort of made sense at the time... And I guess that's why Norton didn't include a defragmentation utility in Norton Utilities 7.0. Incidentally, the only time I ever noticed a difference was in previous OSs when the system was scattered all over the place - starting up did seem much faster when it was defragmented.

sbur

04-24-2002, 06:32 PM

As I understand it, some unix OS's have a built in defragmentation program that kicks in if the disk is more than 3% or so fragmented. This is not, however, universal and Darwin doesn't have it.

As for the need to defragment, I have only anectdotal evidence that it can help. I checked my OS X partition with Norton Utilities 6.0 (booted into OS 9 partition). It was terribly fragmented. The worst I'd ever seen. It was slow to open programs and microsoft word would "lose the connection" to large files. (An actual error I gort when opening 2 MB or so documents.) If you opened the doc and immediately scrolled down, it would disconnect from the document.

I used NU 6.0 to defragment, but did not use disk doctor...I used the disk first aid to diagnose and repair the partition. After defragmentation, I noticed a marked improvement in responsiveness and the disconnecting document problem went away.

While this is not proof that the defragmentation really did it, it's enough for me to believe it had something to do with the performance boost.

I'm still very weary of Norton in OS X as I've heard a number of bad things...but it's what I had, and I had a back up.

I hope that help

Marcwic

04-25-2002, 03:08 AM

I have never degragged a mac volume but it does always speed up my windows machine running NTFS.

I don't think we should have to pay for a defragmenter! Should come free.

MW

Originally posted by sbur
As I understand it, some unix OS's have a built in defragmentation program that kicks in if the disk is more than 3% or so fragmented. This is not, however, universal and Darwin doesn't have it.

As for the need to defragment, I have only anectdotal evidence that it can help. I checked my OS X partition with Norton Utilities 6.0 (booted into OS 9 partition). It was terribly fragmented. The worst I'd ever seen. It was slow to open programs and microsoft word would "lose the connection" to large files. (An actual error I gort when opening 2 MB or so documents.) If you opened the doc and immediately scrolled down, it would disconnect from the document.

I used NU 6.0 to defragment, but did not use disk doctor...I used the disk first aid to diagnose and repair the partition. After defragmentation, I noticed a marked improvement in responsiveness and the disconnecting document problem went away.

While this is not proof that the defragmentation really did it, it's enough for me to believe it had something to do with the performance boost.

I'm still very weary of Norton in OS X as I've heard a number of bad things...but it's what I had, and I had a back up.

I hope that help

JayBee

04-25-2002, 08:15 AM

Have a read at This thread (http://www.macintouch.com/fragmentation.html)

/ * Begin my two cents

The consensus seems to be that OS X "wildly fragments" the hard disk upon installation, and that things stay that way until you boot into OS 9 and "fix" things.

My gut reaction is the good old "OS X is not OS 9, dammit!" :)

I suspect that OS X is quite happy dealing with a "fragmented" drive. The idea that it's a bad thing may fall apart with today's high-speed drives. Also, by scattering data over the whole surface of a drive, and using the "Prebinding" optimisation to locate and flag oft-used files, I suspect you may actually achieve a performance boost, as the drive head will always be statistically the same distance from any file.

However, I could be talking out of my ass. People seem to be reporting performance boosts by defragging their OS X drive from OS 9, but I'm wary of condoning that. Once again, OS X is not OS 9, and this sort of cavalier attitude to doing things the OS 9 way could be problematic further down the line. Bear in mind that OS 9 doesn't use prebinding, so clumping all the files together will help there, but could break OS X in some way.

Also, if it gives THAT much of a performance boost, why aren't Apple publicising it? They've been criticised over OS X's performance, so you'd think a quick fix like this would be high in their knowledge base. As such, a search there for "defrag" doesn't pick up any articles newer than y2k, and they're all concerned with OS 9.

End my two cents */

Has anyone found Apple's line on defragmenting?

Marcwic

04-25-2002, 08:21 AM

The problem if when one file is spilt around a disk, this will always cause a slow down. I am supprised apple haven't included a defragmentor in X, there should have been one in OS9, it's not the type if thing you expect to have to pay for!

I was hoping maybe someone could port a Linux one, but as far as I know the Linux FS (EXT2 i think it's called) defragments as it read/writes or something like that... so they're aren't any Linix defragmentors..!

ashevin

04-29-2002, 01:01 PM

Hi,

The official ext2 utilities do not include a defragger. The filesystem is organized in a way to minimize fragmentation that would be caused by typical disk usage. I have one machine that has been in use for over a year, and fragmentation is less then 5%.

That said, I believe that there exists a project to write a defragger for it. It is in early beta, iirc.

- Avi

P.S. None of the commercial unixes have a defragger, despite having very different filesystems.

thatch

05-15-2002, 02:14 AM

I have noticed very little file fragmentation in OS X after running a defrag and optimize utility, like Speed Disk 6.0.3 booted from OS 9.2.2, until the next time I install or update any system or application stuff. First time OS installs seem to be the heaviest fragged.

But the amount of free space that is fragged happens much more quickly. And when the largest amount of contiguous free space becomes less than 1 GB, then things slow down a lot for me. I can even loose partial DSL connection and see a return of the localhost instead of my ISP's name in my terminal prompt. I do have plenty of ram on my G4 867 too.

I've been a once a week utility kind of guy for over ten years, running Disk Doctor and then Speed Disk after backup is done. I'm not all that thrilled with Disk Doctor lately though because it doesn't work without work around methods because of memory errors which is bad programming IMHO. But Speed Disk has always worked well through the years. Currently, I use it with the OS X profile.

The work I do on the computer doesn't seem all that intensive to me and so I don't think that it should frag as quickly as it does. But reviewing all that I do in a week, I would guess that it is the backup process which causes the most fragmentation. I use psync to an external firewire and to a local partition, run every other day or so depending on what has happened etc...

And that will cause contiguous free space to dwindle very quickly. I notice things start to slow down a bit, more rainbow cursor waiting for apps to launch and the loss of my hostname and some sites on the net that will report server not found, blah blah blah... general finickyness. Free space will report over 500 fragments and the largest chunk of contiguous free space will be something desperately low like under 200 MB's.

Once optimization is complete, things are back to their speedy self until the next time which is seemingly happening sooner that ever lately. I'd say every three or four days I must optimize to prevent the stumbles. And I do all the regular maintenance stuff like the daily, weekly, and monthly cron jobs using anacron plus I update my prebindings after I've done any third party app installations. One other thing... I do shut down every night so my uptime is only a day at a time, not that I think that is causing fragmentation but thought it worth mentioning with regard to the partial loss of the network and free contiguous space stuff.

So, I can't imagine not optimizing the drive ever. I don't think it would continue to work very well for long or so it seems with my setup.

Marcwic

05-15-2002, 02:34 AM

Originally posted by thatch
So, I can't imagine not optimizing the drive ever. I don't think it would continue to work very well for long or so it seems with my setup.

You are defragmenting too often, more than twice a month!

Maybe defrag is putting it all in order, but not leaving free space for other stuff to be put alongside it?

I have speed disk and it puts documents and apps together but leaves a large space in the middle, so other things won't be put at other end of the drive.

thatch

05-15-2002, 02:02 PM

Originally posted by Marcwic
You are defragmenting too often, more than twice a month!

Maybe defrag is putting it all in order, but not leaving free space for other stuff to be put alongside it?

I have speed disk and it puts documents and apps together but leaves a large space in the middle, so other things won't be put at other end of the drive.
Well, I suppose that's a matter of opinion. As I stated, for me, it has become necessary to defrag more than once a week due to the free contiguous space factor. Also as I had stated, the profile I use is the X profile from Donna Armsworthy which leaves the bottom of the drive all free and clear unlike the General default profile which groups with free space in the middle of the drive and is the preferred method for the classic Mac OS, not for OS X.

Marcwic

05-15-2002, 03:26 PM

Right. I have made my OSX profile but how do I load it? When I click select profile in SpeedDisk's menu, the open diolog appears but does not show my profile file! I am boooting from the CD. No mtter what partition i put it on i can't see it to click on it, and load.!

How do I do it? :confused: :confused: :confused: :rolleyes:

thatch

05-15-2002, 05:59 PM

You can't use the custom profile while booted from the CD. But you can run Norton from a spare partition with OS 9 and Norton installed on it.

You put the speed disk profile in your 'Speed Disk Profiles' folder, located inside your 'Norton Tools' folder.

If you are running Norton from a OS 9 partition like I do then the complete path would be something like this:

/Volumes/OS9.2.2/Applications\ $Mac\ OS\ 9$/Utilities/Norton\ SystemWorks\ Folder/Norton\ Utilities\ Folder/Norton\ Tools/Speed\ Disk\ Profiles/os\ x\ profile\ 3.1

Once it is placed there, I don't think you need to choose it from the open dialog. It should appear after a restart and you can just select it from the pull down menu where all the standard profiles are.

Marcwic

05-16-2002, 01:38 AM

Hi thanks for the help with nortan. I don't have it installed in OS9, i am going to install it. I am worried thatbackgound stuff like extentions will slow it down, can I safly startup with minimum extentions (would you reccomend that?) and defrag another partition (same physical disk) - and can I use energy saver to turn of monitor without it going wrong? (I know OS9 isn't too good at multitasking!)

thatch

05-16-2002, 01:54 AM

You can have a minimal OS 9 partition with only the base software on it and Norton. If fact, that's one of the suggested ways to check your partitions without using the Norton CD to boot from a la the Symantec support docs and forums. It works great.

And yes, you can turn the energy saver stuff off so as to not interrupt your lengthy disk checks and optimizations.

Side note:
I've noticed after coming back into the room while Norton has been working for a long while, that the screen is dark with energy saver stuff set to never. But it hasn't caused a problem and just moving the mouse will fire the screen back up again.

rmillikan

06-03-2002, 05:35 PM

This discussion seems to ignore the very different ways that OS9 and OSX load apps into memory. OS9 loads the entire app, making the reading of a contiguous file faster than a fragmented file. OSX (being unix)breaks the app into 4K pages, and reads into memory some number of pages. The program then runs until it refers to an address not in memory. Unix then finds the page having that address and loads that page plus several adjacent pages and the program continues. This process is called a page fault, and is why you can have so many apps in memory at the same time. It's actually more complicated than this, but disk fragmentation is much less of an issue when the system uses programs in 4K chunks. Unix disks always generate a certain amount of fragmentation which the OS controls in ways mysterious to me. But unix people usually never worry about fragmentation.

SSUJoe

06-04-2002, 09:24 PM

I thought I'd put that AppleCare option I paid for to good use.
The tech told me that unless I get errors, the system will take care of itself just fine. I told him about the "clicking" that is heard when opening large files & apps, again, he told me not to worry.

One funny quote, "You're on an Apple, not a Windows machine"

thatch

06-05-2002, 02:34 AM

So long as there are no slow downs, spinning cursors, loss of network, etc... then I would rather not defrag and never have to boot into OS 9 except for maybe running disk repair utilities.

Also, I should add to my previous post that the more fink compiling and app installing I do, the quicker the fragmentation accumulates.

retcynnm

06-14-2002, 09:29 AM

I've found that using Carbon Copy Cloner to back up my OSX partition to an external FW HD,erasing the partition, and then copying it back did an excellent (and free) job of defragging. You then also end up with a bootable back-up.

It also seems that moving my swapfile off of the osx partition has greatly reduced the amount of fragmentation that occurs.I used Swap Cop to move it from the crowded X partition on my TiBook 400 to the much less frequently used os9 partition.

jeffmr

03-31-2003, 12:26 AM

Hello,
I agree with the guys from the UK and Scotland. The way the files are put on the drive and managed is effecient.
I once asked the server administrator's for the computer science department at the University about defragmenting *nixes.
They said you don't have to and there are no utilities for doing it. The only other suggestion they had was to re-install everything.
I could be talking out of my ass too but thats my feeling.
I feel like companies include a defragmenter for psychological consolation more than anything else. If not, let us know.

Cheers, Jeff

tjudd01

04-03-2003, 05:54 PM

Originally posted by thatch
<SNIP>
/Volumes/OS9.2.2/Applications\ $Mac\ OS\ 9$/Utilities/Norton\ SystemWorks\ Folder/Norton\ Utilities\ Folder/Norton\ Tools/Speed\ Disk\ Profiles/os\ x\ profile\ 3.1

Can't that be written a whole lot easier by putting it in quotes?
"/Volumes/OS9.2.2/Applications (Mac OS 9)/Utilities/Norton SystemWorks Folder/Norton Utilities Folder/Norton Tools/Speed Disk Profiles/os x profile 3.1"

(phew, that's quite a path) Maybe a symlink would help
ln -s "/Volumes/OS9.2.2/Applications (Mac OS 9)/Utilities/Norton SystemWorks Folder/Norton Utilities Folder/Norton Tools/Speed Disk Profiles/os x profile 3.1" /SDP3.1

then just using /SDP3.1 should link (aka alias) to the long URL.

HTH

thatch

04-03-2003, 06:23 PM

Originally posted by tjudd01
Can't that be written a whole lot easier by putting it in quotes?
"/Volumes/OS9.2.2/Applications (Mac OS 9)/Utilities/Norton SystemWorks Folder/Norton Utilities Folder/Norton Tools/Speed Disk Profiles/os x profile 3.1"At the time I was directing someone to that location, copy and paste was the easiest way to 'write' that from terminal to the browser. But for an easier 'read', quotes would be a good thing.

Originally posted by tjudd01
(phew, that's quite a path) Maybe a symlink would help
ln -s "/Volumes/OS9.2.2/Applications (Mac OS 9)/Utilities/Norton SystemWorks Folder/Norton Utilities Folder/Norton Tools/Speed Disk Profiles/os x profile 3.1" /SDP3.1

then just using /SDP3.1 should link (aka alias) to the long URL.

HTH I can't imagine why I would want to symlink that for the purpose of posting it here as information to someone in the forum. It just doesn't make any sense at all.

Newbish

04-03-2003, 07:21 PM

In OS X, defragmenting is not as big a deal as in other operating systems. As was mentioned above, UNIX systems generally don't require much attention as far as defragging is concerned.

But it should also be noted that it's not a bad idea to do this periodically, either. It was also noted above that some UNIXes do have defragmenting/disk-optimizing routines. That in itself is a pretty good hint that this activity does have some merit to a UNIX OS.

Essentially, this is simply a matter of housekeeping. If you live in a home with a lot of energetic, boisterous children, you probably have to do a lot more cleaning than someone who lives alone. Well, if you are a heavy-duty poweruser, creating and deleting dozens of files at a time, working graphics or video jobs, it might not hurt to do a defragmentation run once a month. But if you are more of an average user who checks the email once a night, sits down and plays some games, et al, then maybe you only need to think about it once a year or so.

There are two types of "Optimization" that have been brought up in this thread. This means that we need to be careful to differentiate between the two types to avoid confusion among the masses.

"Disk Optimization" is more relevant to this thread. This optimization packs similar types of files, such as applications, closer together. This is pretty much synonymous with defragmenting, which packs the scattered pieces of a single file together.

The other type of optimization is "System Optimization" which is usually performed after a system software installation. This optimization -- as Rob mentioned above -- prebinds the system libraries to the applications so they can find them faster and therefore run better.

Enjoy!

mosch

04-03-2003, 10:44 PM

I come from the unix world, not the Mac world, so this is probably as accurate as all the other guesses.

Nearly all unices do not require defragmentation, because of the file allocation strategies used, which effectively prevent the filesystem from become so fragmented that there are performance problems. Many unices do include defrag utilities, however they're generally there in order to make old DOS users feel comfortable. They're not at all neccessary in any of the Unixes which I've worked with previously (Solaris, SunOS, FreeBSD and Linux).

That being said, excluding the possibility of there being a bug in the defragmentation software, they also can't hurt anything, so it it makes you feel better, feel free to defrag, it probably won't hurt.

I'd be interested in some objective disk i/o performance tests being performed pre and post defragging, in order to see if there's any difference under OS X. I, however, will not be the one performing those tests, because I don't feel like being the one who discovers that the defrag utility did, indeed, have a bug.

NavyIntel007

04-12-2003, 06:35 PM

Mine seems slower after defragmenting. Seems like at boot up it spends a lot of time moving the stuff back to where it used to be.

Accura

04-15-2003, 11:24 AM

I'll install Nortan tomorrow night and do a test for every one. I'll do it on a system volume and a data volume (even tho a system volume is still a data volume, you know what i mean) and post them on here. I'll look back here before I do the tests, see if any one has any suggestions or certain programs i should use, if not ill use the cli using time.

i got a 10 gig 7200 system volume, a 40 gig 5400 system 9 (and music) volume and a 80 gig 7200 data

nexttoyou

04-15-2003, 11:45 AM

Originally posted by Marcwic
Are they any OSX apps, any free ones? Do Macs need defraging? I know my Windows NTFS partition does, what about Macs?

Yes, Mac OS X (HFS+) still needs defragmentation from time to time, depending on your work (e. g. writing and deleting large amounts of files).

Unfortunately there is no such thing as a free defragmentation application available for Mac OS X. However, you could use the donation-ware (uncrippled shareware) Carbon Copy Cloner (http://www.bombich.com/software/ccc.html) if you own a second harddisk (same size or larger). Cloning your disk to a second one, formatting the first one and cloning the content back has the same effect.

artgeek

05-07-2003, 03:00 AM

more reading ->

http://www.experts-exchange.com/Operating_Systems/Macintosh/Q_20426892.html

have fun...

ag

gaelicwizard

05-13-2003, 04:33 PM

I chose not to check "Install OS9 Drivers" when i formatted my hard disk. This is both good for security, and bad for anything that cannot happen in OSX. (like defrag). Is there a way to run defrag on my disk from OS9 (is there a way to 'temporarily' re-enable OS9 support on the disk?)? Thanx, JP

Mikey-San

05-13-2003, 05:44 PM

Originally posted by artgeek
more reading ->

http://www.experts-exchange.com/Operating_Systems/Macintosh/Q_20426892.html

have fun...

ag

YES! YES! YES!

Best post in this thread yet. Volume fragmentation is GOOD, let the word spread.

Here's how I explain it to my customers:

Imagine lining up a hundred $1 bills on the floor and standing at the end of them. If you want the third one, it's easy to get to. If you want the 70th, you gotta go walking. If you put the bills in clumps of, say, ten, scattered around your body, and you need to get to that 70th bill, well, you probably don't have to move as far. And when you go to add or remove bills, you aren't left with massive amounts of empty space you'll have to fill later.

Data can move freely, your drives incur less wear (believe it or not), and your seek times are lower. Go ask your local, psuedo-friendly Unix sysadmin. You know, that guy in the room in the basement who always has those bags of Doritos in his trash can.

YES, we're still using HFS+. YES, HFS+ fragments. NO, that doesn't mean we should be defragging every two weeks (wtf, over) or some archaic junk. (If we have to do that every two weeks, we've a deeper problem.)

mervTormel

05-13-2003, 05:56 PM

hmm, oversimplified analogy, that money thingy. but, anyhow, i don't think defragging even back in pre-X did much good for very long. the first deleted file started the frag dance again.

and with today's faster disk, mox nix. too much trouble, disks are cheap, let 'em frag, outta my way! there's pr0n in them thar hills!

________________
s/you/something smooth and creamy/

yellow

05-13-2003, 06:25 PM

LMAO, well said :D

Mikey-San

05-13-2003, 09:21 PM

Originally posted by mervTormel
hmm, oversimplified analogy, that money thingy. but, anyhow, i don't think defragging even back in pre-X did much good for very long. the first deleted file started the frag dance again.

and with today's faster disk, mox nix. too much trouble, disks are cheap, let 'em frag, outta my way! there's pr0n in them thar hills!

________________
s/you/something smooth and creamy/

I'm still working on the analogy. You gotta watch what you say to users, 'cause you never know what they're going to hear.

This, however, has done wonders for me:

www.mikey-san.net/kitchen.html

Nice sig, by the way. Worn any good t-shirts lately? ;-D

-/-
Mikey-San
"I like my women like I like my bread ... In plastic bags in the refrigerator. No, wait ..."

mervTormel

05-13-2003, 09:57 PM

ah, in that context, i concur. and, the kitchen is a powerful place for analogy.

'scuse me, my biscuits are burning.

Mikey-San

05-14-2003, 07:25 PM

And what hot biscuits they are.

...

dimon

05-20-2003, 03:28 AM

C'mon, you are chickening or what?

HFS and HFS+ both are bazed on binary tree for keepeng tracks of single- or multiclastered files. Logical clasters are based on, say, rigid drive geometry and some other things.

Look:
You bought a new drive. It consists of (for example) two double-sided plates with 16 sectors of 512 bytes per track.
So, you got a hardware clasters of 4x512 bytes in size. Default claster in HFS+ is 4096 bytes. Having interleave 15 you got MINIMUM TWO SPIN ROTATIONS on every logical claster.
Got a first level of disc optimisation, during low-level format.
Here think also about three- sided drives and so on.

Ok, you timed all the things and formatted your drive, and then clattered it with 70000 files of X default install. Keep in mind that HFS plases files in FIRST FREE CLUSTERS. All that tiny holes between clattered files.
Heard something about SWAPFILE?

Hell with it, You just want to work in Photoshop wits 23-layered A2 300 dpi file. Filling all that tiny holes, your drive heads are moving, and seekeng, and moving, ...

Returning to binary tree. Get a nice book and read something about binary tree searching and balancing.

Going to file placement. Plates in drives spin with constant angular speed. Number of sectors per track from center outwards are going bigger, but IN STEPS, so, linear data capasity on tracks IS NOT THE SAME, AS WELL AS READING SPEED.

Going to Norton defrag 6.x.x.
It knows nothing about OS X and threating ALMOST ALL IMPORTAND PARTS of X as DATA, so moves them to
slow end of drive.

Going to numbers. Based on 8500 measuring, but you have no numbers at all.
Bad clustering/partitioning - 3.8 mb/s
standard clustering - 4-5 Mb/s
Optimised clustering/partitioning - up to 7.2 Mb/s
Teoretical limit - 10 Mb/s

You need something like Coctail "Optimise", then booting from OTHER X STARTUP Drive and optimising with Norton 7.x.x.
Keep in mind that Notron 7.0.x is just partially compatible with X 10.2.x, so it's better to have a little older system with Norton to defraging.

Talking about equal access times is dumbiness - SOME FILES ARE MORE EQUAL THEN OTHER®

Dimon.

PS Almost forgot about misterious "UNIX file system, that have no need in defrag" ;)
At first, there IS defrag (man defrag on Solaris or AIX or whatever, for example)
At second, we are speaking about HFS+ or what? Yes, on most Unix systems files are spreaded thry cylinders and sectols from beginning, but AFAIK it was invented (with a bit of data overhead) for minimising losses of data in case of surface damage.

Mikey-San

05-20-2003, 08:07 AM

WTF?

dave@mmu

05-20-2003, 08:13 AM

Drive 10 is OS X based and is really good, if slightly slower than SpeedDisk..

Craig R. Arko

05-20-2003, 08:46 AM

Originally posted by Mikey-San
WTF?

That was my first thought, too. I'm sure there was a message hidden in there some place. Maybe Dimon will clarify it in a future post.

Like Griffman, I haven't defragged an HFS+ volume in years. When it starts to reach that 70% or so full mark where the catalog may be in danger of corruption, it's backup, format and restore time. Faster, safer, and produces a full backup as a happy side effect. IMHO. ;)

It will be of great interest to see what improvements to the filesystem get introduced in Panther. I'm guessing most of the disk utilities will need to be revved again to support it.

GSGM

05-20-2003, 10:57 AM

This is obviously a hotly debated subject.

Some general comments;

1) File Fragementation occurs all the time unless the OS is smart enough not to do it when it writes the file

2) Some OSs have built in tools or kernals that minimize/reduce/clean-up file fragmentation. (OS-X Does not)

3) When a file is read from the disk, it is read a chunk (aka block) at a time. Most files (some applications are exceptions) need to be completly read into RAM, thus several blocks will need to be read consecutivly.

4) In order to keep the overhead time (drive head seek) to a minimum, it makes sense that the blocks of files should be close to each other.

5) There is a certain amount of overhead that is required (CRC checks, block pointer lookups, etc.) that will cause the drive head to rotate pass the next physical block after a block is read. (In fact the "next" block read may in fact be the 3rd, 5th, 10th block physically pass the inital block read)

This is a hardware dependant issue, based upon drive geometry, controller chip, buffer, etc....

Although this is related to the "defrag" question, it is a layer away and should probably be a seperate thread. (ie: If you think the drive is slow seeking, rotating and reading, then defraging isn't your problem.)

Ok, having said (typed) the above, my opinion (and it's just that!) is that defragging makes sense. The caveot is that files should be "grouped" together based on similar needs and that they should be equal distant from the "directory" block.

Using older defrag utilties (Norton for example) you could tell the utility to group by file type (Apps, data, system files, preferences), directory (all the files in a directory together), modifcation time, maximum free space or combinations of the above. This made sense in the pre OS-X days.

In the unix world, files are considered to be data or executables (apps). At the file system level, there is no real distinction between a Jpeg (binary) file and a text file. (ie: in the terminal, you can just as easily type more picture.jpeg as you can note.txt)

OS-X does have some backwards compatible file & creator information that makes it easier for launching preferrred apps, but this is "special" thing.

So, if my conclusions seem logical, what is needed is a tool/utility/kernal that can determine what files are accessed the most (note:system or library files may be used several times but are not changing each time so you cannot use the modification time as a criteria) and place these files close to the directory block (which should probably be in the center of the drive) and then build out from that applications and data files that are used together (ie Appleworks and spreadsheets or Quicken and it's data file) based on the freqency of use and giving preference to the files that don't change (Apps).

Comments?

GSGM

05-20-2003, 11:10 AM

Ok, before someone posts a reply about Piles, I'll mention it. (Note: This is all specutive as I am not aware of the features, specifications, etc.... of Apple's conceptual future OSs)

From what I have heard, Apple played with the idea of "Piles" awhile back and may resurrect it as part of next major OS.

Piles is more of a GUI representation of a grouping of documents. (ex: all of the files and programs needed for a presentation) You can think of it as having multiple directories with nothing but aliases in them that point to the actual files.

Although this sounds like the grouping of docs and apps like I mentioned in the previous post, it doesn't have anything to do with how the files are "arranged" on the drive.

If you have any comments about Piles (or better yet an greater understanding of them) please start another thread.

dimon

05-20-2003, 11:17 AM

Originally posted by Mikey-San
WTF?

Whats wrong exept lang. level?
Can't read or not clear or what?

Dimon

Craig R. Arko

05-20-2003, 11:34 AM

Originally posted by dimon
Whats wrong exept lang. level?
Can't read or not clear or what?

Dimon

Not clear what it is you were getting at specifically. I think you covered a little too much ground in that post.

Were you going after a similar point to what GSGM said few posts later?

dimon

05-20-2003, 12:25 PM

Got it.

In simple:
ANY "Filesystem" is a very complicated combination of hardware and software.
Layered:
1 - REAL drive geometry
2 - Translated or LOGICAL drive geometry (drive presentation to host controller)
3 - .......
.......
II - Logical drive level - mean partitions, and not only visible!
.......
NN - nice icons on desktop

"Defragmentation" in used here sence sticks only with higher levels of "filesystem".

As it's my former speciality, I DO know a bit about this.
And, sorry, go real crazy reading that stuff about "equal access times" and "no defrag for me". And tryed to show that not only "defragmentation" still makes sence, but that there is deeper possibilities of filesystem optimizing.

On kitchen level:
you can get 10000$ by two methods: 10000 bills by 1$ or 1 bill on 10000$.
What is faster to fill and check?
This is "Defrag" in used here sence.

If you HAVE TO fill 10000 forms (somewhat related each with other) its better to keep them organized by alfabet.
It was about B-tree balancing ;)

And if you have a books on shelves (it's closer to me), you better use shelves with some size that corresponds with your arms length and book sizes, yes?
It was about clasters.

Never mind... Just take new gipersuperSerialATARAIDPetabyte drive - and forgot all that crap. For six month.

Dimon

Craig R. Arko

05-20-2003, 12:37 PM

So what would you recommend for a procedure? Is it something based on a % fragmentation trigger, a space used/available trigger or a time trigger?

And how much time would one expect the optimization to consume vs. say, wiping and restoring the drive every 6 months or so? I think I could do that in under 2 hours for a 40 GB partition.

I'm as in favor of efficiency as the next person. :)

dimon

05-20-2003, 12:51 PM

Originally posted by Craig R. Arko
So what would you recommend for a procedure? Is it something based on a % fragmentation trigger, a space used/available trigger or a time trigger?

And how much time would one expect the optimization to consume vs. say, wiping and restoring the drive every 6 months or so? I think I could do that in under 2 hours for a 40 GB partition.

I'm as in favor of efficiency as the next person. :)

Ok, If we'd not talk about X, I'd recommend DiskExpress. I know that they are totally rewritting that really fantastic tool. For X-work, and from their words X filesystem(mean systemworks on filesystem) is mutating too quickly to track that, and they are awaiting for some stable version in near future to release tool.

DiskExpress always was a resident defragmenter that tracks file activity, last/most accessed files, fragmentation level and so on and on idle just defragments file by file. NEVER losing data. NEVER stops. NEVER seen frag more than 0.1 %. Runs only on idle.
This is defragmenter as God invented it.
I'm awaiting for it.

And I have to say that I just can't reformat my drives in way you recommend - I here got a constant dataflow with near 20% of free space everywhere.

Anyway, freshly installed X is not defragged.

Dimon

GSGM

05-20-2003, 01:27 PM

I believe what Dimon is trying to say is that there are some other "issues" that could seriously impact disk performance.

Those issues include the block sizes and arrangment on the drive for example (ie: to save a 1 mb file requires either 250 4k blocks or 1000 1k blocks, depending on what you do one of the choice is better then the other)

While I think these are important, I personally believe they are a little off topic.

{Standing on soapbox for a moment}
If I were to define File Fragmentation it would be this:The condition that occurs when consecutive chunks of the same file are not stored in logically adjacent disk blocks.
{end of soapbox}

An OS will place data anywhere on a drive that does not already have a file written to it. Thus it can (and does) places the first block of data in the center of the drive, the second block at the outer edge and the 3rd block in towards the spindle of the drive for example.

Using the above definition, the file mentioned is said to be fragmented because the chunks are not stored in logicall adjacent blocks. (ie not on the same track nor do they account for the drive spin- reading everyother block as I mentioned in my previous post)

File Fragmentation is something that occurs regaurdless of the block/cluster/claster sizing (Granted, if the block size is equal to or bigger than the file being read only 1 block is needed. But I don't recall too many disk block being multi mega byte in size that can handle an OS update into 1 block)

Therfore, File Fragmentation will occur and will effect performance as a drive will now spend some amount of time moving the head from one end of the drive to the other without reading actual data.

The topic of this thread has more to do with ordering of the blocks such that for a given hardware arrangment (Drive geometry, cluster size, etc...) the amount of time spent moving the drive head to read specific blocks is minimized when compared to the amount of time to completly read the file (ie multiple blocks).

GSGM

05-20-2003, 02:40 PM

Originally posted by Craig R. Arko
So what would you recommend for a procedure? Is it something based on a % fragmentation trigger, a space used/available trigger or a time trigger?

A smarter kernal? :D

But seriously, I like the idea that a tool could run in a background thread or on idle time and defrag on the fly.

As for criteria, the first thing to look at would be the number of consecutive blocks vs. total blocks. (ie Out of the 100 blocks to store a file, 90 are consecutive therefore file A is 90% defragged) I would start with the most fragmented file and then the next most fragmented.

Since there is a possibility that multiple files could have the same level of fragmentation it would be extra cool if there was a weight factor to say that the discontiguous blocks from File B are farther away from the rest of File B then File A's are.

The second criteria should be access time. If I am contantly reading a file (perhaps a library file, preference file, etc..) it could cause significate performance issue simply by having X delay time Y reads.

The third criteria would be modification time. It doesn't make sense to defrag a file that I am going to rewrite in 10 mins.

The above would select which file to work on first. The fact that files could be defragmented in the above mannor automatically would be a great step forward.

The other portion would be to position files such that they are grouped together. (ie when I load Quicken I always read the preferences file and the data file) This would be a much harder thing to do as the data file changes and would grow longer. To make it a defragmented file, it might be nessasary to move it farther away from the app so as not to overwrite the next "group" of files

And how much time would one expect the optimization to consume vs. say, wiping and restoring the drive every 6 months or so? I think I could do that in under 2 hours for a 40 GB partition.

I'm as in favor of efficiency as the next person. :)

2 hours to restore 40Gb? You must have a heck of a backup storage system.

Most users on a reformat would be require to reinstall the OS (and patches to get it back up to speed), reinstall whatever program they used to backup data, then restore data. Depending on what they backed up, (to make backups small they might have just done home directories) they would still have to reinstall Apps too.

Bottom line, I think it is more of a hassle then running a tool manually for an hour every month. (But I still like the idea of something running in the background)

dimon

05-21-2003, 09:56 AM

Hi!

Filesystems are not in kernel, they are more like plugins in Photoshop, so kernel itself have no clue about fragmentation at all. They works on different abstraction levels.

Kinda "object encapsulation" from C++

Dimon

Craig R. Arko

05-21-2003, 10:32 AM

Originally posted by GSGM
A smarter kernal? :D

...

2 hours to restore 40Gb? You must have a heck of a backup storage system.

Yup, another 40 gig partition. :D

Overall I think I'd vote for a smarter filesystem. The sorta thing Dominic Giampaolo might come up with.

dimon

05-21-2003, 12:58 PM

Originally posted by Craig R. Arko
Yup, another 40 gig partition. :D

Overall I think I'd vote for a smarter filesystem. The sorta thing Dominic Giampaolo might come up with.

I think there will be problems with filesystems other than HFS(+). That opinion based on my usage of two partitions - one (BOOT) - UFS and other (storage) - HFS+
Almost any filesystem from UNIX-likes can be ported to X, but...

Do you know that system profiler shows Apple's own UFS implementation as network drive? And it IS correct - from Apple's kernel point of vision.
So, if you are ready for networked drive limitations - you can use any filesystem that exists for, say, FreeBSD. Including such exotic as, say, ReiserFS or something network-distributed from big boys. After little tweaking. ;)
By the way, Apple's UFS is exactly that misterious "UNIX filesystem that have no need in defrag". To get closer to topic.;)

Dimon

sbur

06-16-2003, 12:56 PM

The only time I noticed a performance enhancement after a dfreagmentation (using Norton) was when I first installed OS X.1 and all the apps I normally use (Word, etc.).

Word was having a problem finding all of the fragments of large files, so I kept getting a "Disconnected file" error. So I defragmented (I was amazed at how fragmented the installation of OS X was!). Problem was solved, and the sytem as a whole was much more responsive (2 bounces to launch apps vs like 6-7 bounces). I didn't really benchmark before or after, so this is a qualitative assesment.

That was about two years ago, and I haven't had any performace problems since. I have defragmented a couple of times (routine maintanance habits) and observed to increase (or loss) of performance.

As a side note, I have an OSX partition and a data partition, which houses my users. I do find that defragmenting the data partition can help word, GraphicsConverter, etc. load things faster. But it isn't very significant. (I guess that's my way of targeting file fragmentation without messing with where the apps are in the physical space of the drive.)

vonkas

06-23-2003, 02:39 AM

my 2 cents worth: avoid defragging by preventing fragging! This is how: you need 2 drives. Drive 1 becomes boot drive about 4-6 Gb. Drive 2 should be big - partition in half. One part is for Apps, the other becomes the user dir (edit with netinfo). The user partition also has the desktop but no apps. Because you play a lot with files the user part gets fragmented - who cares! The apps part gets fragmented VERY slowly as it's only changed when de/installing apps - so forget it until you buy a new Mac. And on the boot drive all is more or less static apart from swap files, but they get cleared on the occaisional reboot. Any more questions?