Fault – J.Lo's Tech Blog

Raspberry PI 4 Boot from USB HDD not working GPT

This is more of a note for me than a guide or help for anyone else.

At present the PI4 does NOT support boot from USB, but you can work around this like the old days by using an SD card for the /boot and setting this to load the root from USB Stick/HDD.

However, this only appears to work for me using MBR partition type and just hangs when using GPT. This may be because I’m also encrypting the disk and using headless network boot. But that side does work and the disk/paritions are decrypt and list in /dev/disk/by-label but the OS just will NOT boot for me. Copying everything to the exact same partition layout and labels under MBR and it just works.

It seems a weird problem, as the SD card that’s actually booting and has the kernel is MBR and I don’t see why it would then care, the kernel has to be aware of GPT and as I said lists everything correctly. I can even decrypt and mount during the dropbear network boot bit, but the OS just wont boot. Workaround at present is to place the OS on it’s own drive/stick and use GPT on bigger disks purely for data. I’ve only found this while trying to use a 5Tb disk for both OS and Data. the 2Tb disk I can use MBR on without loosing space so the workaround isn’t really needed, just use MBR.

APT not automatically updating DigitalOcean

I’ve recently noticed a problem on 3 of my Digital Ocean Servers. The APT package lists are not automatically updating every day. I try to keep all servers upto date, and rely on Nagios to inform me when there’s packages needed to be updated and that’s the main reason I noticed something was broken.

The 3 servers in particular are newer builds to the rest of the system, and they dont have near as much installed as the others, so at first I didn’t pay too much attention when other servers were going into warning state on nagios indicating updates but these 3 weren’t. However I would still connect to these servers and run my normal command:-

apt-get update &amp;&amp; apt-get upgrade &amp;&amp; apt-get dist-upgrade &amp;&amp; apt-get autoremove

A few times these servers did install updates and I just thought it must have been my timing, that the package lists hadn’t yet been updated by the cron.daily.

But after this happening a few times, I decided to not run the above and see how long these servers would take for nagios to throw an alert. It never did and that got me a little worried.

Over the last few days I’ve been diagnosing what’s wrong. I started out with making sure cron is working properly. Then kept an eye on the file timestamps

ls -ltrh /var/lib/apt/lists/

Eventually getting to /etc/cron.daily/apt and checking through what was was doing on the working servers compared to the broken ones. I turned on VERBOSE and got a bit of info when running /etc/cron.daily/apt but it seemed to exist quite quicky.

Comparing it to a working server the important bit seemed to be around

+ apt-config shell Debdelta APT::Periodic::Download-Upgradeable-Packages-Debdelta
+ eval
+ [ 1 -eq 0 ]
+ do_cache_backup 0</pre>
On the broken servers I was getting
<pre>+ [ 0 -eq 0 ]
+ [ 0 -eq 0 ]
+ [ 0 -eq 0 ]
+ [ 0 -eq 0 ]

Then it would exit. Further investigating was showing a few settings were being populated on the working servers but not on the broken ones.

So I compared the directory /etc/apt/apt.conf.d/ on both servers an found the following files missing from the broken servers

00aptitude

Aptitude::Get-Root-Command "sudo:/usr/bin/sudo";

10periodic

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";

15update-stamp

APT::Update::Post-Invoke-Success {"touch /var/lib/apt/periodic/update-success-stamp 2&gt;/dev/null || true";};

20archive

APT::Archives::MaxAge "30";
APT::Archives::MinAge "2";
APT::Archives::MaxSize "500";

99update-notifier

DPkg::Post-Invoke {"if [ -d /var/lib/update-notifier ]; then touch /var/lib/update-notifier/dpkg-run-stamp; fi; if [ -e /var/lib/update-notifier/updates-available ]; then echo &gt; /var/lib/update-notifier/updates-available; fi "; };
(reverse-i-search)`apt-': apt-get update &amp;&amp; apt-get upgrade &amp;&amp; apt-get dist-upgrade &amp;&amp; apt-get autoremove

I think the main one being 10periodic but I didn’t fancy spending days/weeks adding each and waiting to see what happened, so I added them all in one go.

Then run /etc/cron.daily/apt again, and this time it’s taken a lot longer to run i.e I’ve written this entire post and it’s still running, as opposed to exiting within a few seconds earlier.

I’m pretty confident that this has now solved my APT package list not automatically updating problem, and providing it has Nagios will start warning on these 3 servers now.

Error ‘glibc detected *** /usr/bin/python: double free or corruption (fasttop)’ on Raspberry Pi using Python

Just a very quick post.
I’m working on a small project I’ve had in mind for a few months, basically pull and image and display it on the tv (there’s more to it, or I’d just settle for RasBMC).

So I’ve been coding it all up using python with pygame to display the image, all fine.

Then I introduced loops to refresh the images, and update the display. I had issues around other code and couldn’t keep looking at the tv while the code was running so I introduced a few mp3’s to play as the images were being refreshed and as the display was updating. All worked well, it sounded like I was on the Enterprise.

Further into coding up different functions, and some more headaches I’d cleared up alot of the minor error’s I had been getting, and was now ready to push the system to speed up the refreshes.

I’d managed to put a few images to refresh every second, and it appeared to be going well.
Then it stopped making a sound, checked and python had crashed with error:
glibc detected *** /usr/bin/python: double free or corruption (fasttop)

So I changed the code I’d been working on (clearly that’s the problem, didn’t have this earlier), but nope didn’t solve it.
Onto google, but this brought up alot of big reports about other things. checked a few, made a few changes. but I’m still getting the problem. Troubling though, if this is going to come down to the fact I’m refreshing alot and it’s getting intensive, it’s going to be a show stopper for this project.

Thankfully I did a little more searching and came across
http://www.raspberrypi.org/forums/viewtopic.php?t=36878&p=308220

Now I dont have the same problem (playing wav’s) but this drew me to the fact I’m playing mp3’s and these have increased in how much I’m playing them and they now overlap alot more than earlier.

So I decided to drop them out. Eureka it’s been running about 15 minutes and not died (previously a few minutes), sad thing is I’m now kinda missing my beeps.
I had hoped to be able to play an alert sound, if certain events happened so I may have to rethink how I can do so without causing crashes.

Anyways there it is in case it’s of help to someone else. I’ll be posting about the project at a later date, once I’ve done a little more with the code and tested it a bit more. It’s my first serious attempt in python, I can make programmers cry at the best of times so I’m expecting people to be able to rip this apart (but I’m actually very interested in getting it stable, so will welcome the criticism/ideas.

Ubuntu Graphics Problems

I ran into this problem a while back on my server but it’s never really affected me as I dont generally use the console.

After running updates a good few months back, my system stopped giving me any output. It wouldn’t load to X or a console and even stopped showing the splash screen and the boot info.

The way I was getting around it was to just use an earlier Kernel version during boot. But tonight while doing some work this little bugger got me again.

After hours working through numerous problems I’ve just left unresolved this one finally needed to be solved. Previously I’d just SSH’d to the machine having no console, started the xlib_shm stuff and up popped the CCTV view and I’d just leave that running, but tonight even that wouldn’t work.

I found the following info while trawling the interweb for fixes:-
The problem seems to start after Kernel 2.6.38-8 (that’s the last one I can boot without problems)
It’s related to the ATI graphics card and drivers
*ERROR* Failed to register bit i2c VGA_DDC was showing up in the syslog

I found info in several bugs saying to add this ‘i2c_algo_bit.bit_test=0’ but didn’t actually say where.
https://bugzilla.novell.com/show_bug.cgi?id=691052
http://lists.freedesktop.org/archives/dri-devel/2011-June/012061.html
http://forums.opensuse.org/english/get-technical-help-here/install-boot-login/459015-no-display-after-online-update-4.html

So I opted to add ‘i2c_algo_bit.bit_test=0’ into the grub config for the lastest kernel, so my grub line now looks like ‘linux /vmlinuz-2.6.38-15-generic root=/dev/mapper/VG-LV_Root ro quiet splash i2c-algo-bit.bit_test=0 vt.handoff=7’

Rebooting kept my splashscreen, and presented me with my console once again. and now my xlib_shm was also working once again.

But as I’m very likely to forget after a bunch more remote updates, I thought it best to make it a little more permanent, so I edited /etc/default/grub and amended the OPTIONS line to read ‘GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash i2c-algo-bit.bit_test=0″‘
a quick run of update-grub and my entire config is updated.
This should survive future updates and keep me with a working console should I need it.

cp: cannot stat `/lib/libnss_*’: No such file or directory

Another Quick note as I keep running into this and every time go hunting for what I did to fix it last time.

When doing update/upgrades I run into the error cp: cannot stat `/lib/libnss_*’: No such file or directory when it’s running update-initramfs

This seems to be because of the dropbear hooks, that I use to decrypt a filesystem remotely.

https://bugs.launchpad.net/ubuntu/+source/dropbear/+bug/834174

Has all the info, but the basic steps for me are:-

nano -w /usr/share/initramfs-tools/hooks/dropbear

search for the line

cp /lib/libnss_* "${DESTDIR}/lib/"

Add x86_64-linux-gnu into the path making the line

cp /lib/x86_64-linux-gnu/libnss_* "${DESTDIR}/lib/"

UPDATE: This is valid for a 64bit system. For a 32bit system the path is more likely to be /lib/i386-linux-gnu/
This would make the line

cp /lib/i386-linux-gnu/libnss_* "${DESTDIR}/lib/"

Then run

update-initramfs -u

and no error.

Asterisk SIP One way Audio

ok, I know what your all thinking just from the title alone.
It’s SIP and One way Audio, it’s going to be a NAT and firewall issue. That was my immediate thought too, but it’s wrong.
To explain a little of the background will probably help.

I’ve been running Asterisk at home now for years, using a Digium 422. One line coming in, and a few Linksys SIP phones.
I’ve been happy with it running away for some time, adding in blacklist functions into the dial plan of incoming calls, blocking withheld numbers into entering their number and building up a list of numbers that are annoying me to reject. I’ve had a few people questioning when they’ve called about it. and even more from visitors whose witnessed the Tivo auto pausing if a call comes in as well as displaying who it is.

So after years of me singing it’s praises (as well as a linux server in general), it’s no-wonder that someone else wanted me to do a similar setup. Enter cousin 🙂

After weeks of searching on ebay, finally someone is selling a digium card. So bought it up quickly.

He’s had a Linux server running for a good few months, and I’ve got it all VPN’d together with my network.
So I installed the Digium card, installed asterisk, and went about configuring it using my config as the basis.
It all worked well, incoming and outgoing calls all fine using the card 2 phones connected and one line.
But obviously, now you’ve got a little you want more. So off he went looking at phones, I have to be fair he found a great deal of linksys phones, 4 of them with power for £40 when most are selling single units at over £20.

So I spend another few hours running network cables to 4 different locations in the house, connect up the phones and go back to configuring asterisk and the phones. Fairly straight forward, run through testing phone to phone etc, all appeared fine.

The following day I had a lovely call saying that their now getting one way audio. Straight away I thought that’s NAT, but as all the phones and asterisk box are on the inside NAT shouldn’t come into it. Still I went back through the SIP config and adjusted lots of it to completely wipe out NAT (I use NAT in my config as I have some connections from SIP providers etc).
More testing, but still the same.
So I started running SIP and RTP traces, I’ve had to read through them bother before and can just about follow what’s going on, but I mainly just look for anything obvious out of place, like IP addresses that are external etc.
Still after going through them, it all looked good in the SIP debug, but the RTP was only showing one side audio on calls that were going in / out over the Line. Calls to other phones were fine.
So I registered one of the phones onto my system and placed a call, that worked fine.
So I come home and registered one of my phones to his system. Placed the call, and again one way audio.

I’m not convinced it’s a NAT problem as my phone would now be running over the VPN to his server and it’s the same server that terminates the VPN. I now started checking out the codecs, but testing using different ones just pointed to it not being codecs. If it couldn’t find common ground the call failed from the start.

It took a few days to get back to as I was onto other stuff, but check after check just showed that everything was fine. But still the one way audio on calls.

I looked again and there was a new version of asterisk released 10.1.3, where I was using 10.1.2 on his server. The 2 fixes in the change were around SIP. So I downloaded the new version, compiled it and installed. Made sure it was all running and then got home to make some test calls.

Still one way audio. So I’m now left with one option. My own server is running an older version of asterisk 1.8.6.0, I uninstalled his asterisk completely and then downloaded the source for 1.8.6.0, compiled and then installed.
Using the same config from the 10.1.3 install threw a few warnings on loading, but it still loaded. Got him to make a test call, he called my mobile and hey presto we can hear each other.

I wanted to update my own asterisk, but after encountering this issue and especially as it’s still present in the latest I think I’ll leave updating my system for a while.

Bottom line, dont just consider One way Audio a NAT problem, and if everything looks to be working correct consider downgrading if there’s no upgrade available.