New Pi HAT board and software

[lrhorer]

I posted a pair of new HAT boards for sale over in the For Sale section, but so far there has been extremely little traffic on it.  I would have expected at least some interest, but judging by the pole I left, only two or three people have even read it.  This project involves a new HAT board along with software for it.  The project is listed here. One feature is the native string port, allowing one to a modest sized string without a much more expensive multi-port Falcon board.  I am certainly not disparaging the Falcon boards.  They are great, and I use five of the 16 port boards myself.  I also have some displays in areas that are not convenient to my Falcon boards. A couple are inside my house, a couple outside, and a few are many miles away.  In these cases, a Falcon board is just not necessary.  The answer here is just a Raspberry Pi Zero or whatever running a string or two directly.  This board offers quite a few more features than that, however, and in fact I think it is a good accessory even if one is using Falcon boards.

In addition to the hardware, I am including what I feel is essential software with the board.  I have some clients who are really rabid about reliability.  This board and its associates software really increase the reliability of the system.  If, like me, you happen to live in a Southern region, and especially if you use FPP during the summer, a fan is more than essential, but even a fan is often not nearly good enough.  I really recommend a Peltier cooler or some similar additional cooling.  For the time being, I am only offering a passive 12 - 24V port for external cooling, but in the future, I may want to add software control to the port

If the temperature does soar for whatever reason on the board, the software will notify the list of email and / or text message parties of the impending situation.  If the temps get really high, the software will shut down the system entirely after notifying everyone on the list.

Similarly, the software keeps track of memory usage, CPU usage, and "disk" usage, and notifies the list if anything starts to get a bit out of hand.  It also notifies the list if the fuse feeding the string blows.

I have been using FPP for many years, now, and while its stability is better now, I still have persistent issues with system lockups on every single Pi running FPP.  It s annoying, at best.  This new board includes an Arduino which provides a watchdog for the Raspberry Pi.  If the Arduino fails to see communications from the Pi, then unless the Pi itself has initiated a shutdown (that heat thing, maybe, you know) the Arduino will take down the 5V power to the Pi for a few seconds and then restore it, inducing the Pi to boot up, again. 'No more having to traipse out into the cold or searing heat to restart a locked up Pi.

What about power failures?  Well, those are covered, too.  The Raspberry Pi also monitors the DC input power, and notifies the list if the power fails.  Wait a minute, Les, are you insane or just stupid?  How can the Pi do anything if it has no power?  Fear not, dear friends.  I am old, but not quite yet senile - I hope.  Included in the design is a Super Capacitor port.  Just hook up a Super Capacitor of an appropriate size, and the Pi will have more than enough time to get the job of notifying the list that power has failed.  (One can attach a battery system, rather than a Super Capacitor, but that is a bit more involved and probably more expensive.)

What else does the software do?  Well, actually, the monitoring software operates in five different modes.

The first is "startup" mode.  Whenever the Pi initially boots up, the monitor waits for 30 seconds for things to settle down, and then sends messages to everyone on the list letting them know the system has just come online.

The second mode is the "check" mode.  This mode is silent to the computer screen, and only does anything if some event or alert is underway.  If some issue is at hand, then that specific problem is noted and sent to the list.  It is designed to be used by FPP, Cron, or whatever, to check the system and send out messages to the list if required. Normally, this would only be if something is amiss, but the user can set up FPP or whatever to notify the list of things via some internal mechanism such as the FPP scheduler.

The third mode is the "print" mode.  This mode outputs all the information to the computer screen.  It only sends specific info to the list if something is wrong.

The fourth mode is the "mail" mode.  This is the reverse of the "print" mode.  All the information is sent out to the list, and nothing appears on the screen.

The fifth mode is the "flash" mode.  This dumps all the info to both the screen and to the list.

The scheduling software runs the monitor every so often, an amount configurable by the user.  If everything is OK, it just goes back to sleep for the prescribed interval.  If something is not OK, then after the monitor sends out the info, the scheduler continues to monitor the issue, but does not send out another message until some different configured amount of time passes.  I image the user would get tired of seeing the same old message every 60 seconds or so.  Once the issue clears, the scheduler will send out one more notification saying everything is OK now.

So what is this list I keep mentioning?  When setting up the software, it will ask the user if he or she wants to set up a list of email addresses and / or text systems to receive communications from the software.  The user can set up as many of either or both as they like.

[JonB256]

For any of my uses for a Pi and PiHat (I use 2 basic hats now, not using DPI), your design is overkill. Then I use a Kulp Pi hat that does use DPI for 8 outputs and Differential outputs. Your target audience may be more commercial than pure hobbyist like me.

[lrhorer]

For any of my uses for a Pi and PiHat (I use 2 basic hats now, not using DPI), your design is overkill. Then I use a Kulp Pi hat that does use DPI for 8 outputs and Differential outputs. Your target audience may be more commercial than pure hobbyist like me.

How is it overkill?  You need less than 1 string? (IOW, no strings at all?)  You are clairvoyant and know automatically when the system freezes? You know without looking when the display goes down it is because of a blown fuse?  I understand you and others may not be old and / or disabled like I am, but I am and so are quite a few of my friends.  It's rather difficult for us to get to the system to restart it.  Still, even for young whippersnappers, getting out to the system and figuring out what is wrong is a pain, especially as compared to having it all taken care of automatically. For that matter, the guy who yells the loudest here (and constantly, mind you) can't ever seem to get to the unit to unplug it for 3 or 4 days, and then complains endlessly about how much the system is down.

[lrhorer]

Here is a screen shot of the print function in its current form:

[Bwinter]

Can you clarify the price per board?  Is it $100 per hat?

[lrhorer]

I just added up the BOMs.  Including shipping and assembly, I believe I can deliver five boards for $70 each.  If only one or two people want one, I will have to charge at least $90.  That is a soft price.  I might be able to better it if I search a bit more for parts.  This includes the Arduino, all headers, soldering, and of course the software.  It does not include a Raspberry Pi, case, 12 - 24V power supply, or a Super Capacitor.  There should be an additional price break at about 50 boards or so.  I will check with my fabricator if interest is sufficient.

[dkulp]

Since you obviously have some capability to write software, why are you not contributing anything back to FPP to make it more reliable or have some of the features you mentioned?   We certainly have people ask about better fan control for FPP and even triggers for the various temperature sensors that are available on the various capes.   That would certainly benefit the community way more than making it specific to an expensive hat with a limited audience.

[lrhorer]

I would love to, and since I have some time on my hands, I was thinking of that very thing.  First off, actually, when / if this software is at a point where it is baked enough for serving up, I would like to integrate it directly into FPP.  It only makes sense.  Please note I am NOT a professional developer, so my code tends to be a bit blocky and inelegant.  At the very least, please be kind when you see it.  I also know nothing at this point of the FPP API, nor have I ever written for an API, so I will no doubt require some hand-holding.  'Not too much, I hope.

I looked through the script repository, and all of them are written in BASH or sh.  While I am very familiar with BASH, it would not be appropriate for this code.  OTOH, I looked through a few plug-ons, and they are all written in C++.  It only makes sense the plug-ins are compiled, but I don't know C++, at all.  I do know a little C.  This app is written in Python 3, which I presume should be fine as a script, but not a plug-in.  I am using some external libraries, so obviously those will need to be loaded as part of the installation.

Moving forward, how can I help?  My writing C++ code might be more trouble for you than it is worth, but I might be able to debug some things, if that will help.  What else?

As to being expensive, a major advantage to this hat is it is LESS expensive than a Falcon V16 or even a Falcon V4 in any situation where the greater display size of an F4 or F16 is not required.  How did I not make that clear?  Secondly, I honestly don't see how this would be considered to be limited.  To my eye, something like this is all but essential.  Even if I were still young and had not encountered the physical issues I have, the last thing I want is to have to deal with persistent failures of an entertainment system, particularly one which is exposed to harsh environment conditions.  I am well acquainted with the costs of this hobby, and it is not cheap.  Many display items cost in the hundreds or even thousands of dollars each.  With labor costs, I have spent well over $20,000 on my system, and I know many people here have spent similar amounts and some have spent much, much more.  To my mind, $100 or so is extremely economical insurance to make sure the system keeps working, or at least notifies the user what went wrong when something as common as a blown fuse or tripped circuit breaker happens.  Believe me, I absolutely understand the value, even necessity, of cutting cost corners wherever practical, but IMO, this is the wrong place to save $75 or so.  Oh, and BTW, this DOES provide a fan, and a high speed one, rather than the 5V whisper fans usually used with a Pi.  The fact the fan does not eat up some of the rather limited power supplied to the Pi nor stress the Pi and its power supply is a plus.  Oh, and since I mentioned the Pi power supply, becasuse this unit runs off the same supply as the display itself, it saves not only the $10 for a Pi power supply, but it also eliminates a wire across the yard / floor and saves a 110V outlet when used as a primary host.

Perhaps I should have stressed that last a bit more, or perhaps I should turn around the nomenclature.  One can, of course think of this as merely a means of providing a single remote string to a holiday display, but a better notion is to think of this as a protection system for the primary host that also provides an output for a single string  (or in the future, maybe two).  It provides cooling at multiple levels, circuit breaker / power failure notification, power simplification, and CPU, memory, and disk monitoring for the meager price of about $65, including the cost of a Super Capacitor (typically under $15).

[darylc]

I'd love to see you contribute these features to FPP, doesn't necessarily have to be via a script or plugin, they might fit in the main code.  This would be much more useful than having to put an arduino with hard to update code on it (eg need USB cable to update) on each cape.

[lrhorer]

What makes you think the Arduin would need to be updated?  Ordinarily, it should not.  That is not to say it can't be, of course, but it only is intended to perform a single function based upon the existence of pulses or loss thereof from the Pi.  These features cannot be handled by the Pi itself, which is why the Arduino is required.

[Bwinter]

Interesting project.  Not sure it aligns with anything I need though (even though my show is 100% PiHat). But I do have some thoughts/questions.

The Arduino power-cycles the Pi when fails to receive any communication from the Pi (the Arduino presume the Pi it's locked-up)?  How does it differentiate a lock-up from a software update?  I would presume the Pi might not respond as normal during the FPP OS upgrades, but you certainly wouldn't want to power-cycle during this time, correct?

As far as monitoring the fuse/power.  Are you referring to the power for the entire board (Pi included), or just the fuses for the output strings?  I presume many people just kill the power to their show frequently (I do every night)—would that trigger a "blown fuse" alert?

[lrhorer]

Interesting project.  Not sure it aligns with anything I need though (even though my show is 100% PiHat). But I do have some thoughts/questions.

The Arduino power-cycles the Pi when fails to receive any communication from the Pi (the Arduino presume the Pi it's locked-up)?  How does it differentiate a lock-up from a software update?  I would presume the Pi might not respond as normal during the FPP OS upgrades, but you certainly wouldn't want to power-cycle during this time, correct?

It's likely the communication will not be interrupted during an OS upgrade, except when the upgrade itself triggers a reboot, which is SOP as far as the Arduino is concerned.  Nonetheless, you are correct it is a very good idea to disable the watchdog while upgrading.  I will provide a small jumper for this very purpose.  When the jumper is removed, the watchdog will no longer be able to kill the power to the RPi.

As far as monitoring the fuse/power.  Are you referring to the power for the entire board (Pi included), or just the fuses for the output strings?  I presume many people just kill the power to their show frequently (I do every night)—would that trigger a "blown fuse" alert?

Since the power for the strings is the same as the power for the Pi, killing it will kill everything after a period of time.  The Pi will take note of the failed input power and report it to the list.  If a SuperCapacitor is used as the power backup, the system will shut off after several seconds.  If a battery is used, the system could remain live for several hours, or even days.  The power fail alarm will mask the fuse alarm, so no fuse alert will go out, but a power failure alert will.  This could be modified to suit the user's requirements.

[Bwinter]

"A small (physical) jumper" that needs to be physically set during software updates?  That sorta negates the usefulness of in-place software updates—which has been a huge benefit the last several years (ie being able to update the software without the need to physically access the controller).

[lrhorer]

"A small (physical) jumper" that needs to be physically set during software updates?  That sorta negates the usefulness of in-place software updates—which has been a huge benefit the last several years (ie being able to update the software without the need to physically access the controller).

I am intending to also  implement a software disable which will send the Arduino into a "Do not disturb" status, which will prevent it from shutting down the Pi, as well.  I suppose I could also create a

"wakeup" signal to get the Arduino out of the loop.  Let me think about it a bit.

[Bwinter]

Have you also considered when the Pi is intentionally SHUTDOWN (as some people do every night)?  If the Arduino doesn't recognize this shutdown, wouldn't it then power/cycle, thus waking up the Pi?  Maybe I missed this earlier in your drscription...

Seems now that it would be a good idea to have a method to easily reprogram the Arduino for software updates (as someone suggested earlier).  Would the Arduino program also be specific to FPP updates (ie could any future FPP updates impact how the Arduino operates)?

Overall, seems this type of feature may introduce more complexity (especially burdensome trying to diagnose if other types of glitches start occurring) than solving any substantial problem.

And as far as the trigger for a fan:  doesn't that already exist natively in the FPP?  I know I can already see each Pi CPU temp (through the FPP interface).  Isn't that info already available to simply trigger a Pi pin (to activates a fan)?  I've never had temp concerns, so it's nothing I've ever looked into—so maybe I'm missing something...