Upgrade problem?

Put support questions regarding OPI here. Examples are hardware related issues and other questions that are specific to OPI.
Post Reply
harrijer
Posts: 7
Joined: Thu Oct 23, 2014 1:14 pm

Upgrade problem?

Post by harrijer » Sun Dec 07, 2014 9:53 pm

Just a heads up to a possible upgrade problem. My opi stopped responding over the week-end. No web interface, no email server.

Root access was still available so I did the following:

ps -ef

UID PID PPID C STIME TTY TIME CMD
root 1 0 0 18:38 ? 00:00:02 /sbin/init
root 2 0 0 18:38 ? 00:00:00 [kthreadd]
root 3 2 3 18:38 ? 00:04:09 [ksoftirqd/0]
root 4 2 0 18:38 ? 00:00:00 [kworker/0:0]
root 5 2 0 18:38 ? 00:00:00 [kworker/0:0H]
root 6 2 0 18:38 ? 00:00:00 [kworker/u2:0]
root 7 2 0 18:38 ? 00:00:47 [rcu_sched]
root 8 2 0 18:38 ? 00:00:00 [rcu_bh]
root 9 2 0 18:38 ? 00:00:00 [migration/0]
root 10 2 0 18:38 ? 00:00:00 [khelper]
root 11 2 0 18:38 ? 00:00:00 [kdevtmpfs]
root 12 2 0 18:38 ? 00:00:00 [kworker/u2:1]
root 14 2 0 18:38 ? 00:00:00 [writeback]
root 23 2 0 18:38 ? 00:00:00 [bioset]
root 25 2 0 18:38 ? 00:00:00 [crypto]
root 27 2 0 18:38 ? 00:00:00 [kblockd]
root 29 2 0 18:38 ? 00:00:00 [khubd]
root 109 2 0 18:38 ? 00:00:00 [rpciod]
root 111 2 0 18:38 ? 00:00:00 [kswapd0]
root 113 2 0 18:38 ? 00:00:00 [fsnotify_mark]
root 115 2 0 18:38 ? 00:00:00 [nfsiod]
root 131 2 0 18:38 ? 00:00:00 [OMAP UART0]
root 199 2 0 18:38 ? 00:00:00 [kpsmoused]
root 204 2 0 18:38 ? 00:00:00 [irq/150-mmc0]
root 518 2 0 18:38 ? 00:00:00 [deferwq]
root 521 2 0 18:38 ? 00:00:00 [irq/86-44e0b000]
root 580 2 0 18:38 ? 00:00:00 [mmcqd/0]
root 623 2 0 18:38 ? 00:00:00 [mmcqd/1]
root 626 2 0 18:38 ? 00:00:00 [mmcqd/1boot0]
root 628 2 0 18:38 ? 00:00:00 [mmcqd/1boot1]
root 644 2 0 18:38 ? 00:00:00 [kworker/0:1H]
root 648 2 0 18:38 ? 00:00:00 [jbd2/mmcblk1p2-]
root 649 2 0 18:38 ? 00:00:00 [ext4-rsv-conver]
root 758 1 0 18:38 ? 00:00:00 upstart-udev-bridge --daemon
root 763 1 0 18:38 ? 00:00:00 /lib/systemd/systemd-udevd --daemon
root 839 2 0 18:38 ? 00:00:00 [ipv6_addrconf]
root 843 1 0 18:38 ? 00:00:00 upstart-file-bridge --daemon
syslog 851 1 0 18:38 ? 00:00:00 rsyslogd
root 895 2 0 18:38 ? 00:00:03 [kworker/0:2]
root 1041 1 0 18:38 ? 00:00:00 cron
root 1055 1 0 18:38 ? 00:00:00 /usr/sbin/dropbear -d /etc/dropbear/dropbear_dss_host_key -r /etc/dropbear/dropbear_rsa_host_key -p 22 -W 65536
root 1081 1 0 18:38 ? 00:00:00 /usr/sbin/opi-backend -d
root 1099 1 0 18:38 ? 00:00:01 php-fpm: master process (/etc/php5/fpm/php-fpm.conf)
www-data 1109 1099 0 18:38 ? 00:00:00 php-fpm: pool www
www-data 1110 1099 0 18:38 ? 00:00:00 php-fpm: pool www
ntp 1184 1 0 18:38 ? 00:00:01 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 104:109
root 1192 2 0 18:39 ? 00:00:00 [kauditd]
root 1346 1055 0 20:10 ? 00:00:00 /usr/sbin/dropbear -d /etc/dropbear/dropbear_dss_host_key -r /etc/dropbear/dropbear_rsa_host_key -p 22 -W 65536
root 1347 1346 0 20:11 pts/0 00:00:00 -bash
root 1407 1347 0 20:22 pts/0 00:00:00 ps -ef



No doevcot etc etc.

Mindful that upgrades were being pushed out, I tried a manual update:

root@opi:/etc# apt-get update
Hit http://repo.op237.se elli InRelease
Hit http://repo.op237.se trusty-stable InRelease
Hit http://repo.op237.se elli/main armhf Packages
Hit http://repo.op237.se trusty-stable/main armhf Packages
Ign http://repo.op237.se elli/main Translation-en
Ign http://repo.op237.se trusty-stable/main Translation-en
E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem.



I ran the dpkg --configure -a but it complained about missing paths.

so:

export PATH=/sbin:/usr/sbin:$PATH

Then started again with apt-get update and this time it ran with no complaints. Followed by apt-get upgrade which (ignoring unable to intialize frontend: Dialog messages) seemed to complete OK.

Did a:

reboot now

And all seems to be working again as it was. Owncloud bug still exists but I think the fix for this was due later.

User avatar
tor
Posts: 120
Joined: Thu Aug 14, 2014 3:42 am
Contact:

Post by tor » Mon Dec 08, 2014 2:58 pm

Hi harrijer,

We/I did a base update of the system. (I.e. updating packages not maintained by us directly) It is strange that the applications stopped worikng. (We do of course test the upgrade on different setups before we release it and we did not encounter any problems such as yours.)

It would be really interesting to know which package it where that where not fully configured/upgraded by the operation.

The Owncloud bugfix is scheduled for the somewhat bigger upgrade of the system including new email server settings. Should hopefully be ready end this week or beginning next week.

/Tor

harrijer
Posts: 7
Joined: Thu Oct 23, 2014 1:14 pm

Post by harrijer » Mon Dec 08, 2014 8:22 pm

Unfortunately - I did not redirect the output of my manual apt-get upgrade :(

However, are the term.log and history.log files in /var/log/apt any use to you?

Jeremy

User avatar
tor
Posts: 120
Joined: Thu Aug 14, 2014 3:42 am
Contact:

Post by tor » Tue Dec 09, 2014 7:23 am

Hi,

The log files are most likely not of any use. When stuff like this happens an easy way to find offending packages is to issue:

dpkg -l | grep -v ^ii

Before proceeding with any fixes. The command would list any packages not cleanly installed.

Thanks anyway for the heads up!

/Tor

sjmcinness
Posts: 1
Joined: Wed Sep 17, 2014 1:36 pm

Post by sjmcinness » Mon Mar 14, 2016 6:26 am

Apologies for digging up an old thread but I just discovered about 350 of the "dpkg was interrupted" messages in my OPI mailbox. For me it seems as though the messages probably started on 29/03/2015.

Here is the output you requested from dpkg:
root@opi:~# dpkg -l | grep -v ^ii
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-================================-============-===============================================================================
iU file 1:5.14-2ubuntu3.3 armhf Determines file type using "magic" numbers
iU libcurl3-gnutls:armhf 7.35.0-1ubuntu2.3 armhf easy-to-use client-side URL transfer library (GnuTLS flavour)
iU libgssapi-krb5-2:armhf 1.12+dfsg-2ubuntu5 armhf MIT Kerberos runtime libraries - krb5 GSS-API Mechanism
iU libk5crypto3:armhf 1.12+dfsg-2ubuntu5 armhf MIT Kerberos runtime libraries - Crypto Library
iU libkrb5-3:armhf 1.12+dfsg-2ubuntu5 armhf MIT Kerberos runtime libraries
iU libkrb5support0:armhf 1.12+dfsg-2ubuntu5 armhf MIT Kerberos runtime libraries - Support library
iU libmagic1:armhf 1:5.14-2ubuntu3.3 armhf File type determination library using "magic" numbers
iU libprocps3:armhf 1:3.3.9-1ubuntu2.1 armhf library for accessing process information from /proc
iU libssl1.0.0:armhf 1.0.1f-1ubuntu2.8 armhf Secure Sockets Layer toolkit - shared libraries
iFR ntp 1:4.2.6.p5+dfsg-3ubuntu2.14.04.1 armhf Network Time Protocol daemon and utility programs
iU ntpdate 1:4.2.6.p5+dfsg-3ubuntu2.14.04.2 armhf client for setting system time from NTP servers
rc php5-xcache 3.1.0-2 armhf Fast, stable PHP opcode cacher
Running the OS-suggested fix:
root@opi:~# dpkg --configure -a
dpkg: warning: 'ldconfig' not found in PATH or not executable
dpkg: warning: 'start-stop-daemon' not found in PATH or not executable
dpkg: error: 2 expected programs not found in PATH or not executable
Note: root's PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin
Adding the paths suggested by harrijer:
root@opi:~# export PATH=/sbin:/usr/sbin:$PATH
root@opi:~# dpkg --configure -a
Setting up libssl1.0.0:armhf (1.0.1f-1ubuntu2.8) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Setting up libprocps3:armhf (1:3.3.9-1ubuntu2.1) ...
Setting up libmagic1:armhf (1:5.14-2ubuntu3.3) ...
Setting up file (1:5.14-2ubuntu3.3) ...
Setting up libkrb5support0:armhf (1.12+dfsg-2ubuntu5) ...
dpkg: error processing package ntp (--configure):
package is in a very bad inconsistent state; you should
reinstall it before attempting configuration
Setting up ntpdate (1:4.2.6.p5+dfsg-3ubuntu2.14.04.2) ...
Setting up libk5crypto3:armhf (1.12+dfsg-2ubuntu5) ...
Setting up libkrb5-3:armhf (1.12+dfsg-2ubuntu5) ...
Setting up libgssapi-krb5-2:armhf (1.12+dfsg-2ubuntu5) ...
Setting up libcurl3-gnutls:armhf (7.35.0-1ubuntu2.3) ...
Processing triggers for libc-bin (2.19-0ubuntu6.5) ...
Errors were encountered while processing:
ntp
NTP doesn't seem too healthy; fortunately re-installing it seems to have resolved that issue:
root@opi:~# apt-get install ntp
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
ntp-doc apparmor
The following packages will be upgraded:
ntp
1 upgraded, 0 newly installed, 0 to remove and 107 not upgraded.
1 not fully installed or removed.
Need to get 374 kB of archives.
After this operation, 1024 B of additional disk space will be used.
Get:1 http://repo.op237.se/opi/ trusty-stable/main ntp armhf 1:4.2.6.p5+dfsg-3ubuntu2.14.04.8 [374 kB]
Fetched 374 kB in 7s (50.3 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline
(Reading database ... 26624 files and directories currently installed.)
Preparing to unpack .../ntp_1%3a4.2.6.p5+dfsg-3ubuntu2.14.04.8_armhf.deb ...
* Stopping NTP server ntpd
...done.
Unpacking ntp (1:4.2.6.p5+dfsg-3ubuntu2.14.04.8) over (1:4.2.6.p5+dfsg-3ubuntu2.14.04.1) ...
Setting up ntp (1:4.2.6.p5+dfsg-3ubuntu2.14.04.8) ...
* Starting NTP server ntpd
...done.
root@opi:~# dpkg --configure -a
root@opi:~#
However the 107 not-upgraded packages suggests that I'm a long way behind. A manual run of apt-get dist-upgrade was successful overall and everything came back up after a reboot. I have the full apt-get console output, if you want/need it.

Scott.

User avatar
tor
Posts: 120
Joined: Thu Aug 14, 2014 3:42 am
Contact:

Post by tor » Mon Mar 14, 2016 4:19 pm

Hi Scott,

Sorry for the problems with OPI. It seems like the upgrade have been interrupted for some reason. (Power failure, loss of network or something similar).

You however seem to have solved all problems yourself according to your post above. You can double check that the cmd "dpkg -l | grep -v ^ii" does not yield any ominous output to make sure you are set otherwise please get back to us!

Best

/Tor

tommyrstone@mail.com
Posts: 6
Joined: Fri Oct 03, 2014 4:51 pm

Post by tommyrstone@mail.com » Tue May 03, 2016 8:39 am

Hello. not completely sure that the upgrade had anything to do with this . It did start happening afterwards though.

I'm was unable to login to email and receive emails on any of my external devices. I was able to get into the other areas like admin, calendar, files, etc until I rebooted opi. Then only able to get into admin. ... then it just logged or kicked me out?

I was unable to ssh into opi -> connection refused.

Then attempted to login and the login screen blinks onto the browser screen a couple of time and disappears and unable to login. I clear browser history for the last hour and the login page appears. I attempt to login and nothing happens. I refresh the page and it tells me invalid username / password. I think this had to do with the self signed cert.

Rebooted again. Then attempted to login.
I was able to login. Emails showed up.
Then it seemed to have logged me out? because it sends me to a similar but slightly different login screen and posts the msg "your session is invalid or has expired." at the bottom.

I attempt to login with this different login screen and it logs me in and everything seems to have went back to normal. :-)

Don't know why. It seems to have self repaired ;-)

The great lesson, that I continue to learn with the opi system, is that if things don't work right ... reboot until they do :) (my experience; may not be the same for others)

Thank you for creating such an amazing product.

Thanks,
trstone

harrijer
Posts: 7
Joined: Thu Oct 23, 2014 1:14 pm

Post by harrijer » Tue May 03, 2016 9:15 am

Actually, this is an ongoing problem with me too.

I have seen most of the same symptoms as you have. I have been in contact with support and we confirmed that what seems to happen is that the authorisation mechanism fails/crashes for some reason. This stops you logging in either via the web interface or via external devices like a mail client.

From my experience, the rest of the system continues working so, for example, mails are still received, you just can't access them.

Because I can't login via the web, rebooting is sometimes brutal as I have to pull the power plug. If I have the ssh password to hand then I do a reboot via the command line which is better.

I have been lazy about following this up as the problem does not appear very often (I have a feeling it happens when I'm hammering the system with a calendar sync but I'm not sure as it's not repeatable). I also use a mail spooling service (EasyDns) so if my opi server goes down, the mails are resent once it is back up which makes me even more lazy. As time is generally not critical in my use case, I live with the occasional outage.

Interesting that this problem is not only on my opi so perhaps a fix can be found. Perhaps we should also start a new message thread (Tor/Pa?).

Cheers,
Jeremy



User avatar
tor
Posts: 120
Joined: Thu Aug 14, 2014 3:42 am
Contact:

Post by tor » Thu May 19, 2016 1:19 pm

Harrijer and trstone,

Sorry for the very late reply. I have totally missed this updated thread, my apologies.

As both of you have experienced there seems to be some bug in the with the components involved in authenticating users. We have observed this our selves and are investigating matters. Harrijer, i think you are right in this happening more frequently when system is under heavier load.

Best!

/Tor

Post Reply