Transterrestrial Musings  


Amazon Honor System Click Here to Pay

Space
Alan Boyle (MSNBC)
Space Politics (Jeff Foust)
Space Transport News (Clark Lindsey)
NASA Watch
NASA Space Flight
Hobby Space
A Voyage To Arcturus (Jay Manifold)
Dispatches From The Final Frontier (Michael Belfiore)
Personal Spaceflight (Jeff Foust)
Mars Blog
The Flame Trench (Florida Today)
Space Cynic
Rocket Forge (Michael Mealing)
COTS Watch (Michael Mealing)
Curmudgeon's Corner (Mark Whittington)
Selenian Boondocks
Tales of the Heliosphere
Out Of The Cradle
Space For Commerce (Brian Dunbar)
True Anomaly
Kevin Parkin
The Speculist (Phil Bowermaster)
Spacecraft (Chris Hall)
Space Pragmatism (Dan Schrimpsher)
Eternal Golden Braid (Fred Kiesche)
Carried Away (Dan Schmelzer)
Laughing Wolf (C. Blake Powers)
Chair Force Engineer (Air Force Procurement)
Spacearium
Saturn Follies
JesusPhreaks (Scott Bell)
Journoblogs
The Ombudsgod
Cut On The Bias (Susanna Cornett)
Joanne Jacobs


Site designed by


Powered by
Movable Type
Biting Commentary about Infinity, and Beyond!

« Memorial Day Thoughts | Main | The Hunters, Not The Hunted »

Fedora Update Update

So I was updating my machine, per the instructions found here, and things were going along swimmingly for a while, until I got the following stretch of error messages:

################################
Other Transactions:
Obsoleting: compat-libstdc++.i386 0:8-3.3.4.2 with compat-libstdc++-296.i386 0:2.96-132.fc4
Obsoleting: cryptsetup.i386 0:0.1-4 with cryptsetup-luks.i386 0:1.0.1-0.fc4
Obsoleting: db4.i386 0:4.2.52-6 with compat-db.i386 0:4.2.52-2.FC4
Obsoleting: db4-devel.i386 0:4.2.52-6 with compat-db.i386 0:4.2.52-2.FC4
Obsoleting: db4-utils.i386 0:4.2.52-6 with compat-db.i386 0:4.2.52-2.FC4
Obsoleting: gcc-g77.i386 0:3.4.2-6.fc3 with compat-gcc-32-g77.i386 0:3.2.3-47.fc4
Obsoleting: gcc-g77.i386 0:3.4.3-22.fc3 with compat-gcc-32-g77.i386 0:3.2.3-47.fc4
Obsoleting: httpd-suexec.i386 0:2.0.52-3 with httpd.i386 0:2.0.54-10.3
Obsoleting: httpd-suexec.i386 0:2.0.52-3.1 with httpd.i386 0:2.0.54-10.3
Obsoleting: kernel-utils.i386 1:2.4-13.1.39 with smartmontools.i386 1:5.33-1.5 Obsoleting: libf2c.i386 0:3.4.2-6.fc3 with libgfortran.i386 0:4.0.2-8.fc4
Obsoleting: libf2c.i386 0:3.4.3-22.fc3 with libgfortran.i386 0:4.0.2-8.fc4
Obsoleting: libtool-libs.i386 0:1.5.6-4 with libtool-ltdl.i386 0:1.5.16.multilib2-3
Obsoleting: libtool-libs.i386 0:1.5.6-4.FC3.2 with libtool-ltdl.i386 0:1.5.16.multilib2-3
Obsoleting: memtest86.i386 0:3.2-1.1.fc3.rf with memtest86+.i386 0:1.55.1-1
Obsoleting: nautilus-media.i386 0:0.8.1-3 with nautilus.i386 0:2.10.0-4
Obsoleting: openoffice.org.i386 0:1.1.2-10 with openoffice.org-math.i386 1:2.0.1.1-5.1
Obsoleting: openoffice.org-i18n.i386 0:1.1.2-10 with openoffice.org-langpack-sv.i386 1:2.0.1.1-5.1
Obsoleting: openoffice.org-libs.i386 0:1.1.2-10 with openoffice.org-core.i386 1:2.0.1.1-5.1
Obsoleting: pdksh.i386 0:5.2.14-30 with ksh.i386 0:20050202-1
Obsoleting: tuxracer.i386 0:0.61-28 with ppracer.i386 0:0.3.1-4.fc4.1
Obsoleting: xscreensaver.i386 1:4.18-4 with xscreensaver-base.i386 1:4.21-4

Is this ok [y/N]: y
Downloading Packages:
libtool-ltdl-1.5.16.multi 100% |=========================| 25 kB 00:00
openoffice.org-langpack-s 100% |=========================| 13 MB 00:39
smartmontools-5.33-1.5.i3 100% |=========================| 257 kB 00:00
microcode_ctl-1.12-1.24_F 100% |=========================| 238 kB 00:01
openoffice.org-langpack-t 100% |=========================| 12 MB 00:33
openoffice.org-langpack-z 100% |=========================| 13 MB 00:38
openoffice.org-langpack-g 100% |=========================| 755 kB 00:02
openoffice.org-langpack-n 100% |=========================| 12 MB 00:33
httpd-2.0.54-10.3.i386.rp 100% |=========================| 935 kB 00:03
openoffice.org-langpack-a 100% |=========================| 4.1 MB 00:13
Traceback (most recent call last):
File "/usr/bin/yum", line 8, in ?
yummain.main(sys.argv[1:])
File "/usr/share/yum-cli/yummain.py", line 136, in main
base.doTransaction()
File "/usr/share/yum-cli/cli.py", line 589, in doTransaction
problems = self.downloadPkgs(downloadpkgs)
File "/usr/lib/python2.3/site-packages/yum/__init__.py", line 382, in downloadPkgs
mylocal = repo.get(relative=remote, local=local, checkfunc=checkfunc)
File "/usr/lib/python2.3/site-packages/yum/repos.py", line 443, in get
checkfunc=checkfunc)
File "/usr/lib/python2.3/site-packages/urlgrabber/mirror.py", line 414, in urlgrab
return self._mirror_try(func, url, kw)
File "/usr/lib/python2.3/site-packages/urlgrabber/mirror.py", line 400, in _mirror_try
return func_ref( *(fullurl,), **kwargs )
File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line 595, in urlgrab
return self._retry(opts, retryfunc, url, filename)
File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line 527, in _retry
return apply(func, (opts,) + args, {})
File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line 583, in retryfunc
fo._do_grab()[root@linux-station home]#

File "/usr/lib/python2.3/site-packages/urlgrabber/grabber.py", line 852, in _do_grab
new_fo.write(block)
IOError: [Errno 28] No space left on device

#######################################

At which point it returned to the root prompt.

What now?

[Update at noon]

Oops! /var is full.

######################################

[root@linux-station home]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda7 10412484 3489804 6393744 36% /
/dev/hda1 101086 8478 87389 9% /boot
none 257904 0 257904 0% /dev/shm
/dev/hda2 100790036 44456020 51214104 47% /home
/dev/hda3 1004052 20080 932968 3% /tmp
/dev/hda5 1004024 1000672 0 100% /var
#######################################

What do I do about that? Any magic tools that can repartition a live drive?


[Update on Monday evening]

OK, I guess the question is, given that (in theory) I've done a partial update from Core 3 to Core 4, but it bombed out part way through, is the machine rebootable? Or do I have to figure out a way to install and run gparted under the current running shell? I don't want to reboot into a Livedisk for repartioning, only to find out that my OS is FUBAR, even with the bigger partition, on reboot.

Posted by Rand Simberg at May 29, 2006 06:43 AM
TrackBack URL for this entry:
http://www.transterrestrial.com/mt-diagnostics.cgi/5541

Listed below are links to weblogs that reference this post from Transterrestrial Musings.
Comments

Buy a Mac?

Posted by philw at May 29, 2006 09:10 AM

No, a Mac would (literally) not serve my needs. Not to mention the fact that I dislike the interface, and don't find them intuitive at all. Go proselytize somewhere else.

Posted by Rand Simberg at May 29, 2006 09:12 AM

Hey Rand

I held off on saying the Mac thing!

Reformat and start over with a clean install?

Sounds like you may have hardware problems screwing with your upgrade. It all started going downhill with your memory problem.

Why don't you find and old hard drive, install Windows and troubleshoot any possible hardware problem with Norton. Linux systems seem to not have as good of hardware troubleshooting software.

As you are going now you are just digging your hole deeper.

Or buy a Mac and install Linux on that box, or if you really like the interface use the B$D Unix and X Windows on the Mac hardware. You don't have to use Aqua.

Dennis

Posted by Dennis Ray Wingo at May 29, 2006 09:17 AM

No, sorry, I'm generally quite happy with AMD and Linux, and I don't have the money to spend on a new Apple that I don't want. I'm just not happy with this particular machine right now.

Posted by Rand Simberg at May 29, 2006 09:24 AM

There's a tool called GParted which will let you edit partitions, including moving and (I think) resizing many of them.

http://gparted.sourceforge.net/index.php

is the home page. There's a Live CD that you can use to work on a system.

Another tool to consider is the Ultimate Boot Disk; it's got a copy of MEMTEST86 on it

http://ubcd.sourceforge.net/

I've played with both, but haven't had to use either in a live recovery situation as yet. I'm moderately gifted with Linux-fu, so if you have questions drop me an email. I'm spambait42c on that yahoo.com thingy. (That should be obscure enough to trick the harvestbots...)

Posted by Glenn at May 29, 2006 11:49 AM

Lets see.

About once every three or four months you end up with problems like this. Hey its your life.

Even if you don't do the Apple thing at least have a hard drive around with another install of Linux or Windows and some troubleshooting tools.

It will save you much pain and suffering the next time this happens to you (in about three or four months).

Dennis


Posted by Dennis Ray Wingo at May 29, 2006 12:38 PM

I have used gparted in a recovery situation.

Some years ago, in a sleep deprived state, I accidentily used dd to blast over the first 1.44 MB of my boot drive! Amazingly, Linux continued to run without error, which allowed me to run gparted and reinstall LILO.

Rand, you're just having a run of bad luck!

OSX, Linux, Windows, whatever - they all need ECC hardware support to conquer defective memory, which your mobo and/or RAM do not have.

As for a full partition, I can't imagine any OS doing better than a "No space left on device" message.

Posted by Eric at May 29, 2006 12:59 PM

About once every three or four months you end up with problems like this.

No, I had a problem a few months ago (which I never really resolved at the time, because I've been busy, and it's not a mission-critical machine right now), and I had a problem now, because I decided to use the holiday break to resolve it, and it turns out they're the same problem--a previously undiagnosed bad memory stick. I've been running Linux in various forms for about eight years, and had few problems. Going out and spending mucho moolah on a machine in which I have no interest would make no sense under the circumstances.

Posted by Rand Simberg at May 29, 2006 03:15 PM

You're making this harder than it needs to be.

type in the following command as root:

du -s /var/*

This will tell you which directories in /var are very full. I'm willing to bet good money that one of the big ones (if not the biggest) will be /var/logs.

Look in /var/logs. You will see many files called

messages, messages.0, messages.1 and so on.

On some versions of Unix they may be called "system" instead of messages.

Delete all messages.* files - they are backups of old system messages and not needed at the moment. Now do this:

du -s /var/log/*

Look for the largest sub-directories. Is one of them especially large? Look at the contents - it's probably logs from one of your applications - the web server, something like that. Delete the contents.

--

If this doesn't free up enough space, then the next thing to look at is uninstalling software you don't use. You'll have to look at the man pages for RPM to figure out how to associate files in /var with the software package that owns them (assuming they aren't obvious like "/var/mysqld" or similar.

If you're worried about deleting something you need, backup /var to a CD-R first.

Posted by Michael heinz at May 29, 2006 05:35 PM

Thanks, I was wondering when I saw that, "Gee, how much of what's currently in /var do I really need? I can certainly live without the logs. That does sound like the most painless way to go. Just clean up the directory, and try to finish the update. Once that's done, I can repartition to prevent future occurences.

Posted by Rand Simberg at May 29, 2006 05:38 PM

I'm afraid, my lack of foresight caused this. I'm very sorry.

I claimed that the risk of having the box unbootable is minimized with yum, and I stand by that. Reboot at will. If it fails, then... I dunno. Dinner maybe?

Yes, yum downloads literally whole distribution into /var/cache/yum before it proceeds upgrading. This means that your /var has to have a lot of free space.

[[Never mind everything until the next bracket... Just so you know what kind of idiot I am, I won't delete it. I wrote all that and _then_ looked at your df output, helpfuly provided...

Naturally, I ran into this problem before, and completely forgot about it. I overcame it in a couple of different cases in a different way. Unfortunately, both are labor intensive.

Method A is to remove worst offenders like OpenOffice.org beforehand, clean them out from /var/cache/yum. Once you see du /var minus du /var/cache/yum going below 55% (of the total!, not just /var; when you remove the size of total decreases), you may proceed. Makes sense, right? This is only good when you can clean that much. Naturally, the list is kept in a file, then missing packages are re-intalled with "yum install", if you need them at all.

Method B is to be used when there's just too much to remove. Like, whole GNOME needs to be killed to reach the free percentage. Then, update package by package, like this: "yum update bash". Do "yum clean packages" in between steps when your /var threatens to overflow (it's inefficient if yum has to re-pull again, but oh well... The key is to update by rough groups, e.g. Gnome, KDE, System utils, Daemons, misc.)

Now that I spelled the options, the whole story stinks. Maybe it was easier to re-install, after all. But at least the RAM is fixed, which is a good thing regardless.]]

Back to real world - your root has 6393744KB free, and /var is only 1004024KB total. The total in the root is only 3489804KB. Isn't the solution obvious?

cd /
mkdir var.new
cp -a var/* var.new
umount var
mv var.new var

That's it. No need to reboot even. You'll need to kill syslog and anyone who's holding /var open (check with fuser -m /var), before you can umount it.

Posted by Pete Zaitcev at May 29, 2006 07:44 PM

EWWWW, what a goofy error. That'll teach you to follow suggestions of strangers on the Internet.

mkdir var.new
cp -a var/* var.new
umount var
rmdir var
mv var.new var

Posted by Pete Zaitcev at May 29, 2006 07:46 PM

EWWWW, what a goofy error.

Yes, I was scratching my head, trying to figure out the method to your madness. Thanks for the suggestion.

Posted by Rand Simberg at May 29, 2006 07:59 PM

Don't use multiple partitions.

Yes, I know that "everyone" advises you to, but how often has doing so helped and how often has it hurt?

Posted by Andy Freeman at May 30, 2006 09:13 AM

Don't use multiple partitions.

For a new setup, I probably woudn't, but this is a legacy system (going back several years, when drives were much smaller), in which I was worried about hard disks filling up due to runaway activities in /var and /tmp.

It reminds me of the old story about Bill Gates and 640K of RAM. No one will ever need more than a gigabyte for /var, right?

Posted by Rand Simberg at May 30, 2006 09:40 AM

The good news is that (if I understand correctly, which is not 100% sure), yum's procedure is:

1) take a list of packages to install

2) figure out what extra packages I need to support those

3) download all the packages in question

4) make a final review to be sure all this will install without contradiction

5) do the actual installations in appropriate order


Since you blew up during step 3, I don't think any actual harm was done to the system.

Posted by Mike Earl at May 30, 2006 10:56 AM


Post a comment
Name:


Email Address:


URL:


Comments: