|
|
10-27-2009, 11:38 AM
|
#1 |
|
Prospect
Join Date: Oct 2009
Posts: 12
|
Spotlight hell since 10.6.1 upgrade
Is anyone else having troubles with spotlight indexing in 10.6.1? I originally did the standard 'upgrade' to snow leopard and had trouble indexing. I decided to backup and wipe the drive but I am still having troubles. Spotlight gets to the stage where it says "Indexing <hostname>... About 46 hours remaining" in the spotlight window but then gives up after filling the disk (i.e. 'du -h' in /.Spotlight-V100 returns 18GB).
-deleting the index with 'mdutil -E /' does not help -disconnecting network drives does not help -deselecting ALL the 'categories' in the spotlight preferences does not help -disk utility lists the permissions ok and the disk is fine -checking the disk with disk warrior indicates no troubles with folders/files/hardware. - no messages for mds/mdworker in syslog until the running low on disk space warning and abort Some debugging (while indexing): Code:
df -h Filesystem Size Used Avail Capacity Mounted on /dev/disk0s2 149Gi 125Gi 23Gi 85% / devfs 119Ki 119Ki 0Bi 100% /dev map -hosts 0Bi 0Bi 0Bi 100% /net map auto_home 0Bi 0Bi 0Bi 100% /home Code:
mdutil -s -a /: Indexing enabled. Code:
echo `pwd`; ls -al /.Spotlight-V100/Store-V1/Stores/5DBC5314-BE76-45B6-B173-44D87D74724E total 4086552 drwx------ 100 root admin 3400 Oct 27 11:28 . drwx------ 3 root admin 102 Oct 27 10:05 .. -rw------- 1 root admin 5083136 Oct 27 11:29 .store.db -rw------- 1 root admin 12582912 Oct 27 11:26 0.indexArrays -rw------- 1 root admin 8192 Oct 27 11:26 0.indexCompactDirectory -rw------- 1 root admin 65792 Oct 27 11:26 0.indexDirectory -rw------- 1 root admin 21846 Oct 27 11:26 0.indexGroups -rw------- 1 root admin 4096 Oct 27 11:27 0.indexHead -rw------- 1 root admin 262144 Oct 27 11:26 0.indexIds -rw------- 1 root admin 8388608 Oct 27 11:26 0.indexPositionTable -rw------- 1 root admin 67108864 Oct 27 11:26 0.indexPositions -rw------- 1 root admin 16777216 Oct 27 11:26 0.indexPostings -rw------- 1 root admin 8388608 Oct 27 11:26 0.indexTermIds -rw------- 1 root admin 0 Oct 27 11:26 0.indexUpdates -rw------- 1 root admin 12582912 Oct 27 11:26 0.shadowIndexArrays -rw------- 1 root admin 5960 Oct 27 11:26 0.shadowIndexCompactDirectory -rw------- 1 root admin 39064 Oct 27 11:26 0.shadowIndexDirectory -rw------- 1 root admin 19575 Oct 27 11:26 0.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:26 0.shadowIndexHead -rw------- 1 root admin 5710512 Oct 27 11:26 0.shadowIndexPositionTable -rw------- 1 root admin 5710512 Oct 27 11:26 0.shadowIndexTermIds -rw------- 1 root admin 28 Oct 27 11:27 indexState -rw------- 1 root admin 0 Oct 27 10:05 journalExclusion -rw------- 1 root admin 727979 Oct 27 11:28 journalLive -rw------- 1 root admin 9580930 Oct 27 11:27 journalSync -rw------- 1 root admin 136562036 Oct 27 10:52 live.0.indexArrays -rw------- 1 root admin 65736548 Oct 27 10:52 live.0.indexCompactDirectory -rw------- 1 root admin 17476 Oct 27 10:52 live.0.indexDirectory -rw------- 1 root admin 36941 Oct 27 11:26 live.0.indexGroups -rw------- 1 root admin 4096 Oct 27 11:27 live.0.indexHead -rw------- 1 root admin 443288 Oct 27 10:52 live.0.indexIds -rw------- 1 root admin 534458002 Oct 27 10:52 live.0.indexPositions -rw------- 1 root admin 83976334 Oct 27 10:52 live.0.indexPostings -rw------- 1 root admin 141 Oct 27 10:52 live.0.indexUpdates -rw------- 1 root admin 36941 Oct 27 11:26 live.0.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:26 live.0.shadowIndexHead -rw------- 1 root admin 140090568 Oct 27 11:19 live.1.indexArrays -rw------- 1 root admin 56083626 Oct 27 11:19 live.1.indexCompactDirectory -rw------- 1 root admin 2056 Oct 27 11:19 live.1.indexDirectory -rw------- 1 root admin 25005 Oct 27 11:23 live.1.indexGroups -rw------- 1 root admin 4096 Oct 27 11:24 live.1.indexHead -rw------- 1 root admin 300056 Oct 27 11:18 live.1.indexIds -rw------- 1 root admin 290064372 Oct 27 11:18 live.1.indexPositions -rw------- 1 root admin 72658274 Oct 27 11:18 live.1.indexPostings -rw------- 1 root admin 144 Oct 27 11:19 live.1.indexUpdates -rw------- 1 root admin 25005 Oct 27 11:24 live.1.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:24 live.1.shadowIndexHead -rw------- 1 root admin 58836456 Oct 27 11:22 live.2.indexArrays -rw------- 1 root admin 31811428 Oct 27 11:22 live.2.indexCompactDirectory -rw------- 1 root admin 2056 Oct 27 11:22 live.2.indexDirectory -rw------- 1 root admin 7847 Oct 27 11:22 live.2.indexGroups -rw------- 1 root admin 4096 Oct 27 11:22 live.2.indexHead -rw------- 1 root admin 94160 Oct 27 11:22 live.2.indexIds -rw------- 1 root admin 103879820 Oct 27 11:22 live.2.indexPositions -rw------- 1 root admin 24478890 Oct 27 11:22 live.2.indexPostings -rw------- 1 root admin 67 Oct 27 11:22 live.2.indexUpdates -rw------- 1 root admin 7847 Oct 27 11:22 live.2.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:22 live.2.shadowIndexHead -rw------- 1 root admin 55029388 Oct 27 11:27 live.3.indexArrays -rw------- 1 root admin 30284520 Oct 27 11:27 live.3.indexCompactDirectory -rw------- 1 root admin 2056 Oct 27 11:27 live.3.indexDirectory -rw------- 1 root admin 7227 Oct 27 11:27 live.3.indexGroups -rw------- 1 root admin 4096 Oct 27 11:27 live.3.indexHead -rw------- 1 root admin 86720 Oct 27 11:27 live.3.indexIds -rw------- 1 root admin 93809425 Oct 27 11:27 live.3.indexPositions -rw------- 1 root admin 22677768 Oct 27 11:27 live.3.indexPostings -rw------- 1 root admin 63 Oct 27 11:27 live.3.indexUpdates -rw------- 1 root admin 7227 Oct 27 11:27 live.3.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:27 live.3.shadowIndexHead -rw------- 1 root admin 24767448 Oct 27 11:28 live.4.indexArrays -rw------- 1 root admin 11501956 Oct 27 11:28 live.4.indexCompactDirectory -rw------- 1 root admin 2056 Oct 27 11:28 live.4.indexDirectory -rw------- 1 root admin 2731 Oct 27 11:26 live.4.indexGroups -rw------- 1 root admin 4096 Oct 27 11:28 live.4.indexHead -rw------- 1 root admin 32768 Oct 27 11:27 live.4.indexIds -rw------- 1 root admin 39232969 Oct 27 11:28 live.4.indexPositions -rw------- 1 root admin 9584949 Oct 27 11:28 live.4.indexPostings -rw------- 1 root admin 1320 Oct 27 11:26 live.4.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:28 live.4.shadowIndexHead -rw------- 1 root admin 6291456 Oct 27 11:29 live.5.indexArrays -rw------- 1 root admin 8192 Oct 27 11:29 live.5.indexCompactDirectory -rw------- 1 root admin 8224 Oct 27 11:26 live.5.indexDirectory -rw------- 1 root admin 2731 Oct 27 11:26 live.5.indexGroups -rw------- 1 root admin 4096 Oct 27 11:27 live.5.indexHead -rw------- 1 root admin 32768 Oct 27 11:26 live.5.indexIds -rw------- 1 root admin 1048576 Oct 27 11:26 live.5.indexPositionTable -rw------- 1 root admin 33554432 Oct 27 11:29 live.5.indexPositions -rw------- 1 root admin 524288 Oct 27 11:26 live.5.indexPostings -rw------- 1 root admin 1048576 Oct 27 11:26 live.5.indexTermIds -rw------- 1 root admin 0 Oct 27 11:26 live.5.indexUpdates -rw------- 1 root admin 3145728 Oct 27 11:26 live.5.shadowIndexArrays -rw------- 1 root admin 1840 Oct 27 11:26 live.5.shadowIndexCompactDirectory -rw------- 1 root admin 2056 Oct 27 11:26 live.5.shadowIndexDirectory -rw------- 1 root admin 10 Oct 27 11:26 live.5.shadowIndexGroups -rw------- 1 root admin 4096 Oct 27 11:26 live.5.shadowIndexHead -rw------- 1 root admin 820360 Oct 27 11:26 live.5.shadowIndexPositionTable -rw------- 1 root admin 820360 Oct 27 11:26 live.5.shadowIndexTermIds -rw------- 1 root admin 5066752 Oct 27 11:26 store.db -rw------- 1 root admin 4 Oct 27 11:26 store.updates -rw------- 1 root admin 0 Oct 27 10:05 tmp.SnowLeopard Code:
ps axcru |sed '1p;/ md/!d' USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 28 12.5 4.5 3393752 189552 ?? Us 6:24PM 27:16.86 mds user 19761 4.5 1.4 2566376 59012 ?? UNs 10:15AM 3:18.62 mdworker _spotlight 20001 0.0 0.1 2466240 3208 ?? SNs 11:27AM 0:00.10 mdworker Code:
lsof -c md |grep -v /System/Library |grep -v Spotlight COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME mds 28 root cwd DIR 14,2 1360 2 / mds 28 root twd DIR 14,2 1360 2 / mds 28 root twd DIR 14,2 1360 2 / mds 28 root twd DIR 14,2 1360 2 / mds 28 root txt REG 14,2 4583 8639167 /usr/lib/mecab/dic/apple/tc/LE/unk.dic mds 28 root txt REG 14,2 5323 8639174 /usr/lib/mecab/dic/apple/ja/LE/unk.dic mds 28 root txt REG 14,2 126464 8633920 /usr/lib/libgermantok.dylib mds 28 root txt REG 14,2 4583 8639160 /usr/lib/mecab/dic/apple/sc/LE/unk.dic mds 28 root txt REG 14,2 262496 8639170 /usr/lib/mecab/dic/apple/ja/LE/char.bin mds 28 root txt REG 14,2 13993264 8635072 /usr/share/icu/icudt40l.dat mds 28 root txt REG 14,2 127012 8639158 /usr/lib/mecab/dic/apple/sc/LE/matrix.bin mds 28 root txt REG 14,2 262496 8639156 /usr/lib/mecab/dic/apple/sc/LE/char.bin mds 28 root txt REG 14,2 1536000 9106271 /private/var/folders/zz/zzzivhrRnAmviuee+++++++++++/-Caches-/com.apple.LaunchServices-0250.csstore mds 28 root txt REG 14,2 262496 8639163 /usr/lib/mecab/dic/apple/tc/LE/char.bin mds 28 root txt REG 14,2 304204 8639165 /usr/lib/mecab/dic/apple/tc/LE/matrix.bin mds 28 root txt REG 14,2 5151552 8639026 /usr/share/langid/langid.inv mds 28 root txt REG 14,2 3646608 8639146 /usr/lib/libmecab.1.0.0.dylib mds 28 root txt REG 14,2 5018101 8633922 /usr/share/germantok/german.index mds 28 root txt REG 14,2 12096866 8639173 /usr/lib/mecab/dic/apple/ja/LE/sys.dic mds 28 root txt REG 14,2 20097804 8639172 /usr/lib/mecab/dic/apple/ja/LE/matrix.bin mds 28 root txt REG 14,2 8419102 8639159 /usr/lib/mecab/dic/apple/sc/LE/sys.dic mds 28 root txt REG 14,2 7495055 8639166 /usr/lib/mecab/dic/apple/tc/LE/sys.dic mds 28 root txt REG 14,2 1054960 8647697 /usr/lib/dyld mds 28 root txt REG 14,2 197427200 8976694 /private/var/db/dyld/dyld_shared_cache_x86_64 mds 28 root 0r CHR 3,2 0t0 297 /dev/null mds 28 root 1 PIPE 0x07164198 16384 ->0x071641fc mds 28 root 2 PIPE 0x07164198 16384 ->0x071641fc mds 28 root 3u KQUEUE count=1, state=0x2 mds 28 root 4r REG 14,2 5652 9106283 /private/var/run/utmpx mds 28 root 5r DIR 14,2 1360 2 / mds 28 root 6 FSEVENT mds 28 root 7r DIR 14,2 1360 2 / mds 28 root 10r DIR 14,2 1360 2 / mds 28 root 15r DIR 14,2 1360 2 / mds 28 root 16r REG 14,2 5151552 8639026 /usr/share/langid/langid.inv mds 28 root 20r DIR 14,2 1360 2 / mds 28 root 24r REG 14,2 5018101 8633922 /usr/share/germantok/german.index mds 28 root 82u unix 0x0715f870 0t0 ->0x0715f900 mdworker 19761 user cwd DIR 14,2 1360 2 / mdworker 19761 user txt REG 14,2 515856 8647973 /usr/lib/libarchive.2.6.2.dylib mdworker 19761 user txt REG 14,2 506560 8868597 /Developer/Library/PrivateFrameworks/JavaKit.framework/Versions/A/JavaKit mdworker 19761 user txt REG 14,2 1245488 8864432 /Developer/Library/PrivateFrameworks/DevToolsSupport.framework/Versions/A/DevToolsSupport mdworker 19761 user txt REG 14,2 1744200 8740613 /usr/share/file/magic.mgc mdworker 19761 user txt REG 14,2 13993264 8635072 /usr/share/icu/icudt40l.dat mdworker 19761 user txt REG 14,2 846768 8862732 /Developer/Library/PrivateFrameworks/DevToolsFoundation.framework/Versions/A/DevToolsFoundation mdworker 19761 user txt REG 14,2 3386784 8862697 /Developer/Library/PrivateFrameworks/DevToolsCParsing.framework/Versions/A/DevToolsCParsing mdworker 19761 user txt REG 14,2 20330768 8862516 /Developer/Library/PrivateFrameworks/DevToolsCore.framework/Versions/A/DevToolsCore mdworker 19761 user txt REG 14,2 1585152 9106223 /private/var/folders/eK/eKwYwlSuH4ekPjlJOqhe4U+++TI/-Caches-/com.apple.LaunchServices-025501.csstore mdworker 19761 user txt REG 14,2 1054960 8647697 /usr/lib/dyld mdworker 19761 user txt REG 14,2 197427200 8976694 /private/var/db/dyld/dyld_shared_cache_x86_64 mdworker 19761 user 0r CHR 3,2 0t0 297 /dev/null mdworker 19761 user 1w CHR 3,2 0t0 297 /dev/null mdworker 19761 user 2w CHR 3,2 0t20969 297 /dev/null mdworker 19761 user 3u KQUEUE count=1, state=0x2 mdworker 19761 user 7r REG 14,2 197806 1553110 /Users/user/Data/etch.cov mdworker 19761 user 8r REG 14,2 197806 1553110 /Users/user/Data/etch.cov Last edited by davep; 10-27-2009 at 02:52 PM. |
|
|
|
10-27-2009, 12:30 PM
|
#2 |
|
Prospect
Join Date: Oct 2009
Posts: 12
|
more...
Code:
find -x / -name \*.mdimporter -exec ls -ldtT {} +
drwxr-xr-x 3 user admin 102 Oct 23 13:18:32 2009 /Applications/OmniFocus.app/Contents/Library/Spotlight/OmniFocusMetadataImporter.mdimporter
drwxr-xr-x 3 user admin 102 Sep 28 04:55:09 2009 /Applications/FontExplorer X Pro 2.0.3.app/Contents/Resources/FontMDI.mdimporter
drwxr-xr-x 3 user admin 102 Sep 28 04:55:09 2009 /Users/user/Library/Spotlight/FontMDI.mdimporter
drwxr-xr-x 3 user admin 102 Sep 28 04:55:06 2009 /Applications/FontExplorer X Pro 2.0.3.app/Contents/Library/Spotlight/FontExplorerXMDI.mdimporter
drwxr-xr-x 3 root wheel 102 Sep 17 19:11:45 2009 /Library/Spotlight/VMware Virtual Machine.mdimporter
drwxr-xr-x 3 user admin 102 Sep 15 08:56:42 2009 /Applications/DEVONthink/DEVONthink Pro.app/Contents/Resources/Spotlight/DEVONthink Spotter.mdimporter
drwxr-xr-x 3 user admin 102 Sep 15 08:56:33 2009 /Applications/DEVONthink/DEVONthink Pro.app/Contents/Library/Spotlight/DEVONthink Spotter.mdimporter
drwxr-xr-x 3 user admin 102 Aug 27 20:27:34 2009 /Applications/OmniPlan.app/Contents/Library/Spotlight/OPMDImporter.mdimporter
drwxr-xr-x 3 user admin 102 Aug 27 14:24:02 2009 /Applications/OmniGraffle Professional 5.app/Contents/Library/Spotlight/OmniGraffle.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 29 01:28:24 2009 /System/Library/Spotlight/Chat.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 28 01:20:29 2009 /System/Library/Spotlight/Mail.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 25 01:35:39 2009 /System/Library/Spotlight/vCard.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 23 22:18:40 2009 /System/Library/Spotlight/Application.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 16 01:53:34 2009 /System/Library/Spotlight/Font.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 15 03:58:43 2009 /System/Library/Spotlight/CoreMedia.mdimporter
drwxrwxr-x 3 root admin 102 Jul 11 03:14:55 2009 /Developer/Applications/Xcode.app/Contents/Library/Spotlight/SourceCode.mdimporter
drwxr-xr-x 3 root wheel 102 Jul 11 03:09:53 2009 /System/Library/Spotlight/Automator.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 30 02:05:50 2009 /System/Library/Spotlight/MIDI.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 30 02:05:48 2009 /System/Library/Spotlight/Audio.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 27 02:08:40 2009 /System/Library/Spotlight/SystemPrefs.mdimporter
drwxrwxr-x 3 root admin 102 Jun 21 20:17:41 2009 /Developer/Applications/Xcode.app/Contents/Library/Spotlight/uuid.mdimporter
drwxr-xr-x 3 user admin 102 Jun 18 17:24:08 2009 /Applications/OmniOutliner Professional.app/Contents/Library/Spotlight/OmniOutlinerMetadataImporter.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 18 01:02:01 2009 /System/Library/Spotlight/Archives.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 15 02:24:36 2009 /System/Library/Spotlight/iPhoto8.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 15 02:24:35 2009 /System/Library/Spotlight/PS.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 15 02:24:34 2009 /System/Library/Spotlight/iPhoto.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 15 02:24:32 2009 /System/Library/Spotlight/Image.mdimporter
drwxr-xr-x 3 root wheel 102 Jun 15 02:24:30 2009 /System/Library/Spotlight/PDF.mdimporter
drwxr-xr-x 3 root wheel 102 May 19 05:17:07 2009 /System/Library/Spotlight/iCal.mdimporter
drwxr-xr-x 3 root wheel 102 May 19 02:18:43 2009 /System/Library/Spotlight/QuartzComposer.mdimporter
drwxr-xr-x 3 root wheel 102 May 19 02:10:16 2009 /System/Library/Spotlight/RichText.mdimporter
drwxrwxr-x 3 root admin 102 May 18 22:12:24 2009 /Library/Spotlight/AppleWorks.mdimporter
drwxr-xr-x 3 root wheel 102 May 18 19:50:48 2009 /System/Library/Spotlight/Bookmarks.mdimporter
drwxrwxr-x 3 root admin 102 May 18 19:49:32 2009 /Library/Spotlight/iWork.mdimporter
drwxrwxr-x 3 root admin 102 Jan 21 05:11:12 2009 /Library/Spotlight/Microsoft Office.mdimporter
Code:
rwsnoop -n mdworker UID PID CMD D BYTES FILE 501 19761 mdworker R 98891 etch081205.cov_005 501 19761 mdworker R 512 etch081205.cov_006 501 19761 mdworker R 98941 etch081205.cov_006 501 19761 mdworker R 512 etch081205.cov_007 501 19761 mdworker R 98925 etch081205.cov_007 501 19761 mdworker R 512 etch081205.cov_008 501 19761 mdworker R 98903 etch081205.cov_008 501 19761 mdworker R 512 etch081205.cov_009 501 19761 mdworker R 98869 etch081205.cov_009 501 19761 mdworker R 512 etch081205.cov_010 501 19761 mdworker R 98977 etch081205.cov_010 501 19761 mdworker R 512 etch081205.cov_011 501 19761 mdworker R 99119 etch081205.cov_011 501 19761 mdworker R 512 etch081205.cov_012 501 19761 mdworker R 99103 etch081205.cov_012 501 19761 mdworker R 512 etch081205.cov_013 501 19761 mdworker R 99571 etch081205.cov_013 501 19761 mdworker R 512 etch081205.cov_014 Code:
rwsnoop -n mds
UID PID CMD D BYTES FILE
0 28 mds R 312
0 28 mds R 151
0 28 mds R 151
0 28 mds R 151
0 28 mds R 1238
0 28 mds R 1242
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 52587 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds R 948 live.5.indexPositions
0 28 mds W 1073 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds R 1711
0 28 mds R 763
0 28 mds R 11435297 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 20 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds R 2332422 tmp.live.5.cmpt..mds.28.compactPayloads1.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
0 28 mds W 65536 tmp.live.5.cmpt..mds.28.compactPayloads2.idx
Last edited by davep; 10-27-2009 at 02:52 PM. |
|
|
|
|
10-28-2009, 06:10 AM
|
#3 |
|
MVP
Join Date: Aug 2006
Posts: 2,281
|
I've had no problems with Spotlight on 10.6.1.
You do have a lot of .mdimporters. Perhaps that's the problem. Certainly try getting rid of this one: /Users/user/Library/Spotlight/FontMDI.mdimporter |
|
|
|
|
10-28-2009, 10:31 AM
|
#4 |
|
Prospect
Join Date: Oct 2009
Posts: 12
|
A surprising number of .mdimporters there just with the base 10.6.x install... install a few applications and all of a sudden you are swimming in them.
I do note that mdworker seems to be spending a lot of time acessing files that it has no hope of reading: 501 19761 mdworker R 98891 etch081205.cov_005 501 19761 mdworker R 512 etch081205.cov_006 I use my laptop for data analysis and have ~40GB of binary/ascii modeling output files. Many are relatively small files with non-sensible file extensions. My latest fear is that spotlight is using a generic text-file importer to scan through the ascii files and index them. |
|
|
|
|
10-28-2009, 10:47 PM
|
#5 | |||||||||||||||||||||||
|
MVP
Join Date: Apr 2002
Posts: 2,395
|
BTW, we can pretty effectively (and permanently) get Spotlight to ignore any particular folder hierarchy by adding a '.noindex' extension onto a folder's name. E.g., /Users/dave/Documents/myData.noindex/ ^ Spotlight will not dork around inside the "myData.noindex" folder (or its subfolders). |
|||||||||||||||||||||||
|
|
|
|
10-29-2009, 11:36 AM
|
#6 | ||||||||||||||||||||||||||||||||||||||
|
Prospect
Join Date: Oct 2009
Posts: 12
|
The KernelEventAgent trips when spotlight runs away and fills the disk with its index. The result is a helpful dialog box telling me my disk is nearly full. Spotlight disables indexing when this occurs which is why the next 2 messages appear. I was really hoping I could toggle a plist setting somewhere and up the verbosity or enable debugging output. As for the 3rd party .mdimporters, I've tried disabling by removing the extension but this didn't help.
I know of the dir.noindex hack but this is less than ideal. I generally store my analysis routines, outputs, plots, and text-based notes alongside the data files/directories. This is a long-standing method of working for me and I've been using nextstep, bsd/*nix in one variant or another for nearly 20 years. Disabling indexing of the entire directory tree means that none of my other files are indexed which defeats the whole 'search' thing. This didn't seem to be a problem for me in 10.5.x. |
||||||||||||||||||||||||||||||||||||||
|
|
|
|
10-28-2009, 05:18 PM
|
#7 |
|
MVP
Join Date: Apr 2002
Posts: 2,395
|
[i don't have Snow-Leo yet, but...] Just wondering: does searching system.log for words like mds, mdworker, mdimport, (etc.), produce any clues?
Here's a mini-script that can be pasted into Terminal to help speed-up such a search: for f in `ls -r /var/log/system.log*`; \ do printf "\n\n\e[1m$f\e[0m >>\n"; \ bzgrep -C 3 --color -e mds $f; done; echo [tweak args to bzgrep as need be] |
|
|
|
|
10-28-2009, 06:43 PM
|
#8 |
|
Prospect
Join Date: Oct 2009
Posts: 12
|
Thanks for the input, syslog was one of the first places I checked. The logs are surprisingly barren with no indication of which importer(s) or file(s) is/are filling the indexes. If anyone knows how to enable some verbose debugging, I'm keen to try it.
Indexing consistently dies with the following in syslog: Code:
Oct 24 02:17:45 Macintosh KernelEventAgent[30]: tid 00000000 type 'hfs', mounted on '/', from '/dev/disk0s2', low disk Oct 24 02:17:45 Macintosh KernelEventAgent[30]: tid 00000001 display_lowdisk_warning: generating lowdisk warning for volume 'Macintosh HD' Oct 24 02:17:45 Macintosh KernelEventAgent[30]: tid 00000000 found 1 filesystem(s) with problem(s) Oct 24 02:17:45 Macintosh mds[28]: (/.Spotlight-V100/Store-V1/Stores/5DBC5314-BE76-45B6-B173-44D87D74724E)(Error) IndexCI in ci_ftruncate:ftruncate(52 /.Spotlight-V100/Store-V1/Stores/5DBC5314-BE76-45B6-B173-44D87D74724E/tmp.merge.19.indexPositions, 8589934592) error:28 Oct 24 02:17:50 Macintosh mds[28]: (/.Spotlight-V100/Store-V1/Stores/5DBC5314-BE76-45B6-B173-44D87D74724E)(Error) IndexGeneral in OuterMerge:Merging failed Oct 24 02:18:45 Macintosh KernelEventAgent[30]: tid 00000001 display_lowdisk_warning: lowdisk warning for volume 'Macintosh HD' removed |
|
|
|
|
10-28-2009, 10:34 PM
|
#9 |
|
MVP
Join Date: Apr 2002
Posts: 2,395
|
That syslog seems to show mds (and "KernelEventAgent") saying the index is funky.
Also, the measurement from post #1... /.Spotlight-V100 returns 18GB ...says that 18 gigs on your HD is in the index? Sicko [way way too high]. I suspect the 3rd-party importers. Apparently inside these apps: OmniFocus, OmniPlan, OmniGraffle, FontExplorer, DEVONthink -- along with others laying around on some library shelf... /Users/user/Library/Spotlight/FontMDI.mdimporter /Library/Spotlight/VMware Virtual Machine.mdimporter Maybe try compressing those mdimporter packages off-disk and nuking the originals. Wipe the current index with sudo rm -fr /.Spotlight* and start over with zero 3rd-party importers. If that works, add them back...slowly... *one* at a time (like one per every 2 or 3 days or something). -- EDIT: curious, does running my mini log-search script turn up *nothing* when run with -e mdworker  Last edited by Hal Itosis; 10-28-2009 at 11:09 PM. Reason: question: mdworker not mentioned in any system.log? |
|
|
|
|
10-29-2009, 12:12 PM
|
#10 |
|
Prospect
Join Date: Oct 2009
Posts: 12
|
Turns out this isn't a 3rd party importer problem but a change in Snow Leopard with respect to 'RichText.mdimporter'. Spotlight is now dead to me
 Consider the file test.csv which contains some numerical data in a common ASCII format (comma delimitated text). Its contents are: Code:
ECLIPSE, 1158904466.03,49,48,-0.04542102,-0.00879937, 0.01820910, 0.03514661, -0.155038, -1.501019 ECLIPSE, 1158904466.28,49,48,-0.04664174,-0.01002009, 0.01897205, 0.03621475, -0.156513, -1.494451 ECLIPSE, 1158904466.53,49,48,-0.04694692,-0.01063045, 0.01897205, 0.03682511, -0.154296, -1.490863 ECLIPSE, 1158904466.67,49,48,-0.04832024,-0.01169858, 0.01912464, 0.03743547, -0.153185, -1.486047 ECLIPSE, 1158904467.03,49,48,-0.04877801,-0.01230894, 0.02004018, 0.03804583, -0.157240, -1.482918 ECLIPSE, 1158904467.28,49,48,-0.04969355,-0.01337708, 0.02049795, 0.03880878, -0.157606, -1.477804 ECLIPSE, 1158904467.53,49,48,-0.05045650,-0.01398744, 0.02080313, 0.03972432, -0.156465, -1.475382 ECLIPSE, 1158904467.67,49,48,-0.05137204,-0.01520816, 0.02126090, 0.04033468, -0.157335, -1.469743 ECLIPSE, 1158904468.03,49,48,-0.05228758,-0.01597111, 0.02126090, 0.04125022, -0.154351, -1.466945 ECLIPSE, 1158904468.28,49,48,-0.05305053,-0.01749701, 0.02202385, 0.04170799, -0.157572, -1.459768 Code:
kea:~ user$ uname -a
Darwin Kea.nrhome 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386
kea:~ user$ mdimport -pn -d 2 test.csv
2009-10-29 12:00:32.971 mdimport[26218:10b] Imported '/Users/user/test.csv' of type 'dyn.ah62d4rv4ge80g650' with no plugIn.
2009-10-29 12:00:32.973 mdimport[26218:10b] Attributes: {
"_kMDItemFinderLabel" = <null>;
"com_apple_metadata_modtime" = 278524825;
kMDItemContentCreationDate = 2009-10-29 12:00:25 -0400;
kMDItemContentModificationDate = 2009-10-29 12:00:25 -0400;
kMDItemContentType = "dyn.ah62d4rv4ge80g650";
kMDItemContentTypeTree = (
"public.data",
"public.item"
);
kMDItemDisplayName = {
"" = "test.csv";
};
kMDItemKind = {
"" = Document;
da = Dokument;
de = Dokument;
es = Documento;
fi = Dokumentti;
it = Documento;
ja = "\U66f8\U985e";
ko = "\Ub3c4\Ud050\Uba58\Ud2b8";
nb = Dokument;
pl = dokument;
pt = Documento;
"pt-PT" = Documento;
ru = "\U0414\U043e\U043a\U0443\U043c\U0435\U043d\U0442";
sv = Dokument;
"zh-Hans" = "\U6587\U7a3f";
"zh-Hant" = "\U6587\U4ef6";
};
}
Total processing time spent in importer plug-ins: 0.047622 seconds for 1 files
Top 1 most expensive files for importer plug-ins:
1 /Users/user/test.csv: 0.047622 seconds
Code:
[mollymawk] ~ > uname -a
Darwin mollymawk.nrhome 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386
[mollymawk] ~ > mdimport -pn -d 2 test.csv
(Info) Import: magic_file returned "ASCII text" for path "/Users/user/test.csv" of type "public.comma-separated-values-text"
(Info) Import: magic identified contents of file at path "/Users/user/test.csv" as text; using text importer
(Info) Import: Import '/Users/user/test.csv' type 'public.comma-separated-values-text' using '/System/Library/Spotlight/RichText.mdimporter'
2009-10-29 12:03:19.254 mdimport[816:903] Imported '/Users/user/test.csv' of type 'public.comma-separated-values-text' with no plugIn.
2009-10-29 12:03:19.295 mdimport[816:903] Attributes: {
":MD:kMDItemSeedLastUsedDate" = 1;
"_kMDItemFinderLabel" = "<null>";
"com_apple_metadata_modtime" = 278524985;
kMDItemAuthors = "<null>";
kMDItemComment = "<null>";
kMDItemContentCreationDate = "2009-10-29 12:03:05 -0400";
kMDItemContentModificationDate = "2009-10-29 12:03:05 -0400";
kMDItemContentType = "public.comma-separated-values-text";
kMDItemContentTypeTree = (
"public.comma-separated-values-text",
"public.delimited-values-text",
"public.text",
"public.data",
"public.item",
"public.content"
);
kMDItemCopyright = "<null>";
kMDItemCreator = "<null>";
kMDItemDisplayName = {
"" = "test.csv";
};
kMDItemEditors = "<null>";
kMDItemKeywords = "<null>";
kMDItemKind = {
"" = Sheet;
};
kMDItemOrganizations = "<null>";
kMDItemSubject = "<null>";
kMDItemTextContent = " ECLIPSE, 1158904466.03,49,48,-0.04542102,-0.00879937, 0.01820910, 0.03514661, -0.155038, -1.501019\n ECLIPSE, 1158904466.28,49,48,-0.04664174,-0.01002009, 0.01897205, 0.03621475, -0.156513, -1.494451\n ECLIPSE, 1158904466.53,49,48,-0.04694692,-0.01063045, 0.01897205, 0.03682511, -0.154296, -1.490863\n ECLIPSE, 1158904466.67,49,48,-0.04832024,-0.01169858, 0.01912464, 0.03743547, -0.153185, -1.486047\n ECLIPSE, 1158904467.03,49,48,-0.04877801,-0.01230894, 0.02004018, 0.03804583, -0.157240, -1.482918\n ECLIPSE, 1158904467.28,49,48,-0.04969355,-0.01337708, 0.02049795, 0.03880878, -0.157606, -1.477804\n ECLIPSE, 1158904467.53,49,48,-0.05045650,-0.01398744, 0.02080313, 0.03972432, -0.156465, -1.475382\n ECLIPSE, 1158904467.67,49,48,-0.05137204,-0.01520816, 0.02126090, 0.04033468, -0.157335, -1.469743\n ECLIPSE, 1158904468.03,49,48,-0.05228758,-0.01597111, 0.02126090, 0.04125022, -0.154351, -1.466945\n ECLIPSE, 1158904468.28,49,48,-0.05305053,-0.01749701, 0.02202385, 0.04170799, -0.157572, -1.459768\n";
kMDItemTitle = "<null>";
}
Total processing time spent in importer plug-ins: 0.369842 seconds for 1 files
Top 1 most expensive files for importer plug-ins:
1 /Users/user/test.csv: 0.369842 seconds
10.5.x Code:
Kea:~ user$ uname -a
Darwin Kea.nrhome 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386
Kea:~ user$ echo '0.2110 1.0652 0.1857 1.0848 -1.2509 -0.5182 0.6170 -1.4375 1.4643 -1.9356' > foobar.ccc
Kea:~ user$ mdimport -pn -d 2 foobar.ccc
2009-10-29 12:09:40.570 mdimport[26246:10b] Imported '/Users/user/foobar.ccc' of type 'dyn.ah62d4rv4ge80g25d' with no plugIn.
2009-10-29 12:09:40.572 mdimport[26246:10b] Attributes: {
"_kMDItemFinderLabel" = <null>;
"com_apple_metadata_modtime" = 278525373;
kMDItemContentCreationDate = 2009-10-29 12:09:33 -0400;
kMDItemContentModificationDate = 2009-10-29 12:09:33 -0400;
kMDItemContentType = "dyn.ah62d4rv4ge80g25d";
kMDItemContentTypeTree = (
"public.data",
"public.item"
);
kMDItemDisplayName = {
"" = "foobar.ccc";
};
kMDItemKind = {
"" = Document;
da = Dokument;
de = Dokument;
es = Documento;
fi = Dokumentti;
it = Documento;
ja = "\U66f8\U985e";
ko = "\Ub3c4\Ud050\Uba58\Ud2b8";
nb = Dokument;
pl = dokument;
pt = Documento;
"pt-PT" = Documento;
ru = "\U0414\U043e\U043a\U0443\U043c\U0435\U043d\U0442";
sv = Dokument;
"zh-Hans" = "\U6587\U7a3f";
"zh-Hant" = "\U6587\U4ef6";
};
}
Total processing time spent in importer plug-ins: 0.049228 seconds for 1 files
Top 1 most expensive files for importer plug-ins:
1 /Users/user/foobar.ccc: 0.049228 seconds
Code:
[mollymawk] ~ > uname -a
Darwin mollymawk.nrhome 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386
[mollymawk] ~ > echo '0.2110 1.0652 0.1857 1.0848 -1.2509 -0.5182 0.6170 -1.4375 1.4643 -1.9356' > foobar.ccc
[mollymawk] ~ > mdimport -pn -d 2 foobar.ccc (Info) Import: magic_file returned "ASCII text" for path "/Users/user/foobar.ccc" of type "dyn.ah62d4rv4ge80g25d"
(Info) Import: magic identified contents of file at path "/Users/user/foobar.ccc" as text; using text importer
(Info) Import: Import '/Users/user/foobar.ccc' type 'dyn.ah62d4rv4ge80g25d' using '/System/Library/Spotlight/RichText.mdimporter'
2009-10-29 12:11:24.280 mdimport[839:903] Imported '/Users/user/foobar.ccc' of type 'dyn.ah62d4rv4ge80g25d' with no plugIn.
2009-10-29 12:11:24.284 mdimport[839:903] Attributes: {
":MD:kMDItemSeedLastUsedDate" = 1;
"_kMDItemFinderLabel" = "<null>";
"com_apple_metadata_modtime" = 278525476;
kMDItemAuthors = "<null>";
kMDItemComment = "<null>";
kMDItemContentCreationDate = "2009-10-29 12:11:16 -0400";
kMDItemContentModificationDate = "2009-10-29 12:11:16 -0400";
kMDItemContentType = "dyn.ah62d4rv4ge80g25d";
kMDItemContentTypeTree = (
"public.data",
"public.item"
);
kMDItemCopyright = "<null>";
kMDItemCreator = "<null>";
kMDItemDisplayName = {
"" = "foobar.ccc";
};
kMDItemEditors = "<null>";
kMDItemKeywords = "<null>";
kMDItemKind = {
"" = Document;
de = Dokument;
en = Document;
fr = Document;
};
kMDItemOrganizations = "<null>";
kMDItemSubject = "<null>";
kMDItemTextContent = "0.2110 1.0652 0.1857 1.0848 -1.2509 -0.5182 0.6170 -1.4375 1.4643 -1.9356\n";
kMDItemTitle = "<null>";
}
Total processing time spent in importer plug-ins: 0.210648 seconds for 1 files
Top 1 most expensive files for importer plug-ins:
1 /Users/user/foobar.ccc: 0.210648 seconds
Does anyone want to guess what happens when you try and crudely index many, many text files containing seemingly random numbers? Last edited by davep; 10-29-2009 at 01:45 PM. |
|
|
|
|
10-29-2009, 05:56 PM
|
#11 | ||||||||||||||||||||||||||||||||||||||||||||||
|
MVP
Join Date: Apr 2002
Posts: 2,395
|
Nice work! Well, it did seem to be the fault of some importer... so naturally i just assumed (with all the 3rd-party choices on your drive) that it would *not* be an Apple-branded one. 
Why not try grabbing a /System/Library/Spotlight/RichText.mdimporter out of 10.5 and stick it in Snowy then? Looks like it has never been updated on my disk since the initial 10.5.0 install (running 10.5.8 now). Code:
$ pkgutil -v --file-info /System/Library/Spotlight/RichText.mdimporter
volume: /
path: System/Library/Spotlight/RichText.mdimporter
pkgid: com.apple.pkg.Essentials
pkg-version: 10.5.0.1.1.1192168948
install-time: Thu Nov 29 14:08:51 2007
uid: 0 (root)
gid: 0 (wheel)
mode: 40755 (drwxr-xr-x )
|
||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
10-29-2009, 06:35 PM
|
#12 |
|
Prospect
Join Date: Oct 2009
Posts: 12
|
It is actually mdimport which is now detecting the files as ascii instead of just relying on common dot extensions or file attributes:
mdimport -pn -d 2 foobar.ccc (Info) Import: magic_file returned "ASCII text" for path "/Users/user/foobar.ccc" of type "dyn.ah62d4rv4ge80g25d" (Info) Import: magic identified contents of file at path "/Users/user/foobar.ccc" as text; using text importer (Info) Import: Import '/Users/user/foobar.ccc' type 'dyn.ah62d4rv4ge80g25d' using '/System/Library/Spotlight/RichText.mdimporter' also see 'man mdimporter' DESCRIPTION mdimport is used to test Spotlight plug-ins, list the installed plug-ins and schema, and re-index files handled by a plug-in when a new plug-in is installed.The old RichText.mdimporter is capable of parsing any ascii file but it was not being called into action in 10.5.x. I does appear possible to downgrade /usr/bin/mdimport but I'm not sure if there are any other ramifications. I've written a condensed, summary version of all this at: http://forums.cnet.com/5208-6126_102...hreadID=366184 Last edited by davep; 10-29-2009 at 07:57 PM. Reason: typos |
|
|
|
|
10-29-2009, 11:17 PM
|
#13 | |||||||||||||||||||||||
|
MVP
Join Date: Apr 2002
Posts: 2,395
|
Hmm, that word 'magic' sounds a lot like the file command (man file). I wonder if Spotlight has its own internal version of magic... or, if mdimport is simply calling the /usr/bin/file utility and/or consulting its config file[?]. (i.e., man file talks about using /usr/share/file/magic for info... and also states that: "in addition file will look in $HOME/.magic.mgc, or $HOME/.magic for magic entries"). So... *if* mdimport is now calling /usr/bin/file [or reading its magic data], then maybe your situation could be alleviated by tweaking the ... . . . oh hell, i don't know. I don't even have 10.6, so i can't test out anything. Anyway... you seem to be doing fine without any help.  Keep up the good work! -- WHOA! Bingo... from post #1: lsof -c md |grep -v /System/Library |grep -v Spotlight mdworker 19761 user txt REG 14,2 1744200 8740613 /usr/share/file/magic.mgc edit #2: Ewww, it's a big old ugly binary data file. Hopefully there's some mechanism to supersede it. And there seems to be several of them: Code:
$ ls -l /usr/share/file/magic* -rw-r--r-- 1 root wheel 444126 Sep 23 2007 /usr/share/file/magic -rw-r--r-- 1 root wheel 2664288 Sep 23 2007 /usr/share/file/magic.mgc -rw-r--r-- 1 root wheel 31192 Sep 23 2007 /usr/share/file/magic.mime -rw-r--r-- 1 root wheel 121792 Sep 23 2007 /usr/share/file/magic.mime.mgc Last edited by Hal Itosis; 10-30-2009 at 12:02 AM. |
|||||||||||||||||||||||
|
|
|
|
10-30-2009, 12:09 AM
|
#14 | |||||||||||||||||||||||
|
Prospect
Join Date: Oct 2009
Posts: 12
|
Nice catch! Its been a very long time since I've seen file and magic used for anything. Oh the horror. Will take a look at the .mgc files tomorrow. EDIT: indeed it is pulling from the old magic database. Feed mdimport an ascii VRML file and reports: (Info) Import: magic_file returned "VRML 1 file" for path "/Users/user/foobar.new" of type "dyn.ah62d4rv4ge8063p1" Best of all, it doesn't insert anything into kMDItemTextContent. Last edited by davep; 10-30-2009 at 12:28 AM. |
|||||||||||||||||||||||
|
|
|
|
10-30-2009, 02:08 AM
|
#15 | |||||||||||||||||||||||
|
Moderator
Join Date: Jan 2002
Location: Montreal
Posts: 29,448
|
I see that the folder "file-23/file/magic/Magdir" in the Darwin source code tree for Leopard contains a whole bunch of ASCII files which appear to be the source for this binary database. So this could potentially be rebuilt if desired.
__________________
hayne.net/macosx.html |
|||||||||||||||||||||||
|
|
|
 |
| Tags |
| indexing, snow leopard, spotlight |
| Thread Tools | |
| Display Modes | Rate This Thread |
|
|
Hybrid Mode
