2016年10月22日土曜日

crash during system upgrade

I tried to fix crash at last full system update.
Firstly I verified and reinstalled modules at the system update. I found linux-firmware and linux have problem and reinstalled both. After reinstall, linux-firmware seems to be fixed but linux has still errors. As I noticed other module seems to have problems, I think my system is not clean. Anyway, the first problem missing nvidia firmware is cleared so I reboot system. First thing is remove nvidia package and use nouveau as driver.

log

  • make new archiso archlinux-2016.10.01-dual.iso
  • arch-chroot
  • switch to nouveau
    • install nouveau and mesa-libgl pacman -S --needed xf86-video-nouveau mesa-libgl
    • remove nvidia pacman -R nvidia
    • also remove nvidia-utils pacman -R nvidia-utils
    • added nouveau in /etc/mkinitcpio.conf
    • regenerage initramfs mkinitcpio -p linux
  • reboot
    it seems nouveau driver loaded successfully and login screen show up but it’s same as nvidia driver case, it freeze and I can do nothing other than force shutdown.

So verify and reinstall last upgraded packages didn’t solve problem.
Next I should try is this
https://wiki.archlinux.org/index.php/pacman#pacman_crashes_during_an_upgrade

The procedures

  1. Boot using Arch installation media. Preferably use a recent media so that the pacman version match / is newer than the system.
  2. Mount the system’s root filesystem mount /dev/sdx /mnt as root, and check the mount has sufficient space with df -h
  3. If the system uses default database and directory locations, you can now update the system’s pacman database and upgrade it via
    pacman --root /mnt --cachedir=/mnt/var/pacman/pkg -Syyu
    as root.
  4. After the upgrade, one way to double-check for not upgraded but still broken packages: find /mnt/usr/lib -size 0
  5. Followed by reinstall of any still broken package via
    pacman --root /mnt --cachedir=/mnt/var/pacman/pkg -S package

Tried this but not solved I think.
The problem is my system missing some system files. How to find those files and owner of those packages?
this post hinted me. Just pacman -Qkk verify all packages.

output log
/home/y4suyuki/log/pacman_crash_all.txt
http://sprunge.us/MVTj

rm conflict files empty not owned by any packages and reinstall.

corrupt log

warning: java-runtime-common: /usr/lib/jvm/default (Symlink path mismatch)
warning: java-runtime-common: /usr/lib/jvm/default (Modification time mismatch)
warning: java-runtime-common: /usr/lib/jvm/default-runtime (Symlink path mismatch)
warning: java-runtime-common: /usr/lib/jvm/default-runtime (Modification time mismatch)error: error while reading file /var/lib/pacman/local/man-pages-4.08-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/openssh-7.3p1-2/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/python2-pyparsing-2.1.10-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/python2-setuptools-1:28.3.0-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/ristretto-0.8.1-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/sudo-1.8.18-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/systemd-sysvcompat-231-4/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/texinfo-6.3-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/xf86-input-evdev-2.10.4-1/mtree: Unrecognized archive format
error: error while reading file /var/lib/pacman/local/xfce4-notifyd-0.3.3-1/mtree: Unrecognized archive format

reinstalled

  • lvm2 /etc/lvm/lvmlocal.conf.pacnew
  • pacman-mirror /etc/pacman.d/mirrorlist.pacnew
  • man-pages
  • sudo /etc/sudoers.pacnew
  • systemd-sysvcompat
  • xf86-input-evdev
  • openssh /etc/ssh/ssh_config.pacnew /etc/ssh/sshd_config.pacnew
  • texinfo
  • xfce4-notifyd
  • python2-setuptools
  • python2-pyparsing
  • ristretto

After reinstalled these packages and rebooted then … it worked!

Written with StackEdit.

2016年10月19日水曜日

Arch system upgrade problem

Problem in upgrading system

Yesterday I did regular system upgrade using pacman. What I did was simply run

$ sudo pacman -Syu

Then suddenly my monitor is blackout. I confused and just shutdown pc manually pressing shutdown button.
Right after I tried to reboot but reboot failed and error message.

arch linux error premature end of file vmlinuz-linux

I did chroot to system using liveCD. I found that /boot/vmlinuz-linx is empty file. I searched on the web for solution and checked pacman log /var/log/pacman.log

So I chroot to system from liveCD and install linux

$ pacman -S linux
$ sudo mkinitcpio -p linux

This create /boot/vmlinuz-linux file but still boot process is failing and display show starting version 231.

The questions

  • What is the starting version 231 means and where is it of boot process
    System just stuck and I can’t enter command. How to boot from liveCD? Enter BIOS to change setup of UEFI
  • What’s wrong about my system?
    • where is the log files to check
      most log files are under /var/log also check dmesg and journalctl
    • how to get more informative debug messages
      remove quiet parameter in grub

Log

  • boot from liveUSB
  • mount disks based on my setting
  $ mount /dev/sda2 /mnt
  $ mount /dev/sda1 /mnt/boot
  $ mount /dev/sda3 /mnt/usr
  $ mount /dev/sdb1 /mnt/var
  $ mount /dev/sdb2 /mnt/tmp
  $ mount /dev/sdb4 /mnt/home
  $ swapon /dev/sdb3
  • chroot
    $ arch-chroot /mnt /bin/bash
  • search around log files
  • $ Xorg :0 -configure
    failed no device to configure error
  • pacman -Syu
  • mkinitcpio -p linux
    output seems successful
  • reboot but no luck. still stuck ‘starting version 231’
  • pastebin logs
Oct 15 12:59:19 arch-corsair kernel: fb: switching to nouveaufb from EFI VGA
Oct 15 12:59:19 arch-corsair kernel: Console: switching to colour dummy device 80x25
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: NVIDIA GM206 (126020a1)
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: bios: version 84.06.2f.00.71
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: disp: dcb 15 type 8 unknown
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: loading /lib/firmware/nvidia/gm206/gr/sw_nonctx.bin failed with error -22
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: Direct firmware load for nvidia/gm206/gr/sw_nonctx.bin failed with error -22
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: gr: failed to load gr/sw_nonctx
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: gr ctor failed, -22
* `journalctl -p 3 --since="2016-10-15 12:57:00" | curl -F 'sprunge=<-' http://sprunge.us`
http://sprunge.us/YDcP
-- Logs begin at Thu 2015-11-12 23:00:07 JST, end at Sat 2016-10-15 12:59:38 JST. --
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: gr: failed to load gr/sw_nonctx
Oct 15 12:59:19 arch-corsair kernel: nouveau 0000:01:00.0: gr ctor failed, -22
Oct 15 12:59:19 arch-corsair kernel: sp5100_tco: I/O address 0x0cd6 already in use
Oct 15 12:59:33 arch-corsair systemd[1]: Failed to start Light Display Manager.
* `dmesg --level=err,warn | curl -F 'sprunge=<-' http://sprunge.us`

http://sprunge.us/UYBj

[    0.000000] ACPI: RSDP 0x00000000BDE6C000 000024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 0x00000000BDE6C078 00006C (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: FACP 0x00000000BDE72540 00010C (v05 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI BIOS Warning (bug): Optional FADT field Pm2ControlBlock has zero address or length: 0x0000000000000000/0x1 (20150619/tbfadt-654)
[    0.000000] ACPI: DSDT 0x00000000BDE6C178 0063C2 (v02 ALASKA A M I    00000000 INTL 20051117)
[    0.000000] ACPI: FACS 0x00000000BDE75080 000040
[    0.000000] ACPI: APIC 0x00000000BDE72650 00008E (v03 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: FPDT 0x00000000BDE726E0 000044 (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: MCFG 0x00000000BDE72728 00003C (v01 ALASKA A M I    01072009 MSFT 00010013)
[    0.000000] ACPI: HPET 0x00000000BDE72768 000038 (v01 ALASKA A M I    01072009 AMI  00000005)
[    0.000000] ACPI: SSDT 0x00000000BDE727A0 001158 (v01 AMD    POWERNOW 00000001 AMD  00000001)
[    0.000000] ACPI: BGRT 0x00000000BDE738F8 000038 (v00 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: IVRS 0x00000000BDE73930 0000C0 (v01 AMD    RD890S   00202031 AMD  00000000)
[    0.000000] ACPI: BGRT 0x00000000BDE739F0 000038 (v00 ALASKA A M I    01072009 AMI  00010013)
[    0.002735] ACPI: All ACPI Tables successfully acquired
[    0.332605] ACPI: Executed 2 blocks of module-level executable AML code
[    0.334809] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20150619/hwxface-580)
[    0.334813] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20150619/hwxface-580)
[    2.976826] r8169 0000:04:00.0: can't disable ASPM; OS doesn't have ASPM control
[    2.981807] MXM: GUID detected in BIOS
[    3.096851] nouveau 0000:01:00.0: Direct firmware load for nouveau/nv126_fuc409c failed with error -2
[    3.096860] nouveau 0000:01:00.0: Direct firmware load for nouveau/fuc409c failed with error -2
[    3.096862] nouveau E[  PGRAPH][0000:01:00.0] failed to load fuc409c
[    3.097031] nouveau E[     DRM] Pointer to flat panel table invalid
[    3.097952] nouveau W[     DRM] unknown connector type 70
[    3.097987] nouveau W[     DRM] failed to create encoder 1/8/0: -19
[    3.097988] nouveau W[     DRM] Unknown-1 has no encoders, removing
[    3.100856] nouveau E[   PFIFO][0000:01:00.0] unsupported engines 0x00000001
[    3.100954] nouveau E[     DRM] failed to create kernel channel, -22
[ 1348.911390] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[+0.00s] DEBUG: Logging to /var/log/lightdm/lightdm.log
[+0.00s] DEBUG: Starting Light Display Manager 1.18.3, UID=0 PID=468
[+0.00s] DEBUG: Loading configuration dirs from /usr/share/lightdm/lightdm.conf.d
[+0.00s] DEBUG: Loading configuration dirs from /usr/local/share/lightdm/lightdm.conf.d
[+0.00s] DEBUG: Loading configuration dirs from /etc/xdg/lightdm/lightdm.conf.d
[+0.00s] DEBUG: Loading configuration from /etc/lightdm/lightdm.conf
[+0.00s] DEBUG: Using D-Bus name org.freedesktop.DisplayManager
[+0.00s] DEBUG: Registered seat module xlocal
[+0.00s] DEBUG: Registered seat module xremote
[+0.00s] DEBUG: Registered seat module unity
[+0.01s] DEBUG: Monitoring logind for seats
[+0.01s] DEBUG: New seat added from logind: seat0
[+0.01s] DEBUG: Seat seat0: Loading properties from config section Seat:*
[+0.01s] DEBUG: Seat seat0: Starting
[+0.01s] DEBUG: Seat seat0: Creating greeter session
[+0.01s] DEBUG: Seat seat0: Creating display server of type x
[+0.01s] DEBUG: Could not run plymouth --ping: Failed to execute child process "plymouth" (No such file or directory)
[+0.01s] DEBUG: Using VT 7
[+0.01s] DEBUG: Seat seat0: Starting local X display on VT 7
[+0.01s] DEBUG: DisplayServer x-0: Logging to /var/log/lightdm/x-0.log
[+0.01s] DEBUG: DisplayServer x-0: Writing X server authority to /run/lightdm/root/:0
[+0.01s] DEBUG: DisplayServer x-0: Launching X Server
[+0.01s] DEBUG: Launching process 474: /usr/sbin/X :0 -seat seat0 -auth /run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
[+0.01s] DEBUG: DisplayServer x-0: Waiting for ready signal from X server :0
[+0.01s] DEBUG: Acquired bus name org.freedesktop.DisplayManager
[+0.01s] DEBUG: Registering seat with bus path /org/freedesktop/DisplayManager/Seat0
[+0.01s] WARNING: Error getting user list from org.freedesktop.Accounts: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.Accounts was not provided by any .service files
[+0.01s] DEBUG: Loading user config from /etc/lightdm/users.conf
[+0.01s] DEBUG: User y4suyuki added
[+0.15s] DEBUG: Process 474 exited with return value 1
[+0.15s] DEBUG: DisplayServer x-0: X server stopped
[+0.15s] DEBUG: Releasing VT 7
[+0.15s] DEBUG: DisplayServer x-0: Removing X server authority /run/lightdm/root/:0
[+0.15s] DEBUG: Seat seat0: Display server stopped
[+0.15s] DEBUG: Seat seat0: Stopping; greeter display server failed to start
[+0.15s] DEBUG: Seat seat0: Stopping
[+0.15s] DEBUG: Seat seat0: Stopping session
[+0.15s] DEBUG: Seat seat0: Session stopped
[+0.15s] DEBUG: Seat seat0: Stopped
[+0.15s] DEBUG: Required seat has stopped
[+0.15s] DEBUG: Stopping display manager
[+0.15s] DEBUG: Display manager stopped
[+0.15s] DEBUG: Stopping daemon
[+0.15s] DEBUG: Seat seat0: Stopping session
[+0.16s] DEBUG: Exiting with return value 1
  • added /etc/sysctl.d/99-sysctl.conf with line
    kernel.sysrq = 1 for shortcut keys but shortcut keys not working
  • install nvidia $ pacman -S nvidia to switch video card driver
  • remove nouveau from /etc/mkinitcpio.conf
  • run mkinitcpio -p linux
  • reboot. the login screen show up! But.. it feezed. I typed first character of password then I can’t type in and also mouse don’t move.
  • added hook for nvidia in pacman.d
  • install nvidia-libgl pacman -S nvidia-libgl remove mesa-libgl
  • posted to arch linux forum
    https://bbs.archlinux.org/viewtopic.php?id=218376
  • reinstall linux-firmware pacman -S linux-firmware
    Then pacman -Qkk linux-firmware emit no error message
  • reinstall linux pacman -S linux it successfully reinstalled but pacman -Qkk linux show same error message.

The last post from fsckd in forum suggest see pacman crashes during an upgrade might be what happened. pacman -Qkk linux didn’t fix linux and some package (like nodejs which own no file and /usr/bin/node is empty) seem to be corrupt.

Written with StackEdit.