News:

LATEST RELEASE:  FPP 7.5 - Download from here - https://github.com/FalconChristmas/fpp/releases/tag/7.5

+-+-

+-User

Welcome, Guest.
Please login or register.
 
 
 
Forgot your password?

+-Site Stats

Members
Total Members: 16396
Latest: rafelog927
New This Month: 0
New This Week: 0
New Today: 0
Stats
Total Posts: 132038
Total Topics: 16364
Most Online Today: 132
Most Online Ever: 7634
(January 21, 2020, 02:14:03 AM)
Users Online
Members: 0
Guests: 76
Total: 76

Intermittent fpp hangs (7.2.x, 7.3.5)

Started by aggieNick02, December 06, 2023, 08:20:17 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

aggieNick02

My setup was solid last year, but upgrading to 7.2 then 7.3 this year I am getting an intermittent hang every 4 days or so. The last two times the web UI was not responsive and the show had frozen - unfortunately each time I had to reboot without trying to investigate more via ssh.

Looking at the logs for both hangs, each hang had the last (non-noise) line in /var/log/syslog:

FPP systemd[1]: Starting phpsessionclean.service - Clean php session files...

(The log is also filled with constant "pcf85363_rtc_read_time: error -121" errors every 2 seconds, but those just seem like log noise.)

I just updated to 7.3.6 right now - the last hang was on 7.3.5 tonight.

Any ideas?

jtrpop

Just happened to me. Viewer IMed me saying no sound with lagging lights. When I got to check it FPP was inaccessible and lights all dark. Rebooted it and it came back up. Never had these issues before. On 7.3.5 and see 7.3.6 was released. Will need to update after show is done.
Jason Rasmussen
www.facebook.com/LightsOnSymphonySt
Virtual Christmas Summit 2021 - Optimizing Your Audio and FM Transmission https://bit.ly/vcsaudio2021

aggieNick02

Interesting. Any idea how many days your FPP was running before it went down? The one change in 7.3.6 that caught my eye was fixing lots of memory leaks. I installed atop so if it happens again I can see if memory was climbing to exhaustion before it froze.

CaptainMurdoch

Do you know if you could ssh or was it totally offline? 
-
Chris

jtrpop

@CaptainMurdoch When I realized my show was dark, I tried to SSH but could not. Inaccessible via web too. I don't recall if I could ping it or not. Pulled power, plugged in, it came back up.

It had been running a few days on 7.3.5 without issue, but last week (older FPP, maybe 7.3.4?) I did have some laggy lights once, audio was fine though. Reboot fixed it. I'm using exact same hardware and sequences from last year. Only change was two new Sandisk SSDs (one for my Pi master, the other my Pi remote) flashed fresh with 7, upgraded to 7.3.5, backup imported from a FPP backup. So really only change is FPP version and the new SD cards. Have never had these issues before.

Guy IMed me and told me what sequence it started glitching out on if I could find it in the logs if I know what to look for.

Thanks
Jason Rasmussen
www.facebook.com/LightsOnSymphonySt
Virtual Christmas Summit 2021 - Optimizing Your Audio and FM Transmission https://bit.ly/vcsaudio2021

jtrpop

Upgraded to 7.3.6. Let's see how that goes   ;D
Jason Rasmussen
www.facebook.com/LightsOnSymphonySt
Virtual Christmas Summit 2021 - Optimizing Your Audio and FM Transmission https://bit.ly/vcsaudio2021

aggieNick02

Quote from: CaptainMurdoch on December 06, 2023, 10:16:22 PMDo you know if you could ssh or was it totally offline? 
Unfortunately I don't know. First of the two times I was leaving the house and only had time to unplug and replug, and last night my son did the same as I had just got in the shower when he noticed it. If it happens again, I'll see if I can ssh or get ping responses.

The log from last night continued after the hang until power reset (nest cam audio let me discern when it happened). But after the hang the log only had the read_time noise entries. So at least that time, the system was not completely hung.

Happy to share any logs, etc, that might help

jtrpop

7.3.6 and it ran for a few days, but then borked again tonight. Lights frozen at one spot, no audio. I could SSH, but it did seem poky. Web GUI was accessible, but said "FPPD Daemon is not running" and reboot did nothing. Had to pull power to reset the Pi. I'll try 7.4 next, might swap the SD (which is brand new), but I'm hearing lots of other people with similar issues with 7.x in different Facebook groups.
Jason Rasmussen
www.facebook.com/LightsOnSymphonySt
Virtual Christmas Summit 2021 - Optimizing Your Audio and FM Transmission https://bit.ly/vcsaudio2021

CaptainMurdoch

7.3.5 and prior had some memory leaks which have now been fixed.  If you can still reproduce in 7.4, then we'll need to take a look at the fppd.log or crashlog.
-
Chris

falknerlights

As others have posted, I have had several FPP lockups or crashes since upgrading to 7.x. I had no lockups last year. I have already had one instance of it occurring in 7.4 (last night). Could get to the web browser and it said FPPD daemon not running. Restarting FPP did not solve it, had to reboot the Pi via the web browser. My system has one master Pi driving sACN to the rig and two remote Pi's with P5 panels. Only lockups appear to have been on the master. Also noted that after the reboot only the last scheduled event ran (I have an event that runs a few minutes before the main show to run a script to take control from a Mosaic MSC running permanent lights, which did not run after the reboot). The master Pi is kept powered up 24/7 and is on a UPS, while the remotes are powered down each night.

Poporacer

Quote from: falknerlights on December 12, 2023, 01:31:59 PMI have already had one instance of it occurring in 7.4 (last night). Could get to the web browser and it said FPPD daemon not running.
How about the log files to give the developers something meaningful to look at? And the date and time that this happened.
If to err is human, I am more human than most people.

falknerlights

Which logs are they looking for, and what is the best way to get them to them?

Poporacer

Go to Content Setup>File Manager and then the Logs tab. Click on the Zip button...It will download all the log files.
If to err is human, I am more human than most people.

falknerlights

Attached. I updated to 7.4 on 12/9 and the first time the show didn't run under 7.4 was 12/11 (had to reboot Pi). On 12/12 the take control event did not run, but the main show did. I am also getting a lot of can't ping controller messages during the show, but not lagging or loss of control was observed on the display.You cannot view this attachment.

Poporacer

Quote from: falknerlights on December 14, 2023, 03:01:28 PMAttached.
Thanks...a few things. I see a TON of network errors with the controllers coming on and off line which could be some sort of networking issue. I would look at that!
Also there are some errors with the FPPMon plugin that looks like it is network related as well? I would remove the plugin until you get the networking resolved.

You are also getting an error on the  White Christmas - 1s Leadin.mp3 file. I would suggest deleting this file. Re-encoding the original and then re-uploading it to make sure you didn't get a corrupt file somehow
If to err is human, I am more human than most people.

Support FPP

+- Recent Topics

Top-notch –°asual Dating Legitimate Girls by DClark5079
Today at 03:36:16 AM

FPP After Hours Music Plugin has been updated by MikeKrebs
February 21, 2024, 10:24:39 PM

Kiosk mode touchscreen by Poporacer
February 19, 2024, 08:57:25 AM

reviving a bbb on an octoscroller by Poporacer
February 18, 2024, 11:25:33 AM

External WiFi Adapter and Antenna by JonB256
February 16, 2024, 08:25:13 AM

troubles getting UCS2904/TM1814 to work properly by ocupmoc
February 12, 2024, 11:15:53 AM

F16v4s keep freezing up by Zcarpenter
February 12, 2024, 08:02:46 AM

xLights in Ubuntu 22.04 by LedMutt
February 07, 2024, 08:48:43 PM

FPP 8 early alpha pre-release feedback by LedMutt
February 07, 2024, 08:39:23 PM

Why a custom/minimized bootstrap implementation in fpp? by jcross
February 07, 2024, 12:03:14 PM

Powered by EzPortal
Powered by SMFPacks Menu Editor Mod