Author Topic: FPP v2.x plugin integration  (Read 2169 times)

Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #15 on: January 15, 2019, 10:15:42 PM »
I'm thinking now that since you have to be online to install a new plugin, that we should just retrieve the info from the internet similar to the script repository.  This will ensure the most up to date plugin info is loaded and will get the FPP developers out of the business of maintaining SHA's in pluginData.inc.php.

Here's what I am thinking (and what I have mostly coded and tested already)  :)

- FPP repo has a simple fpp/js/pluginList.js file that contains an array of known plugins along with a link to their pluginInfo.json info file.  The pluginInfo format is described at the bottom of this post.  The pluginInfo.json files are stored in the plugin repositories, not in the FPP repository.
- When  you load the plugin installer page, the UI will look at both the list of installed plugins and the list of plugins in the pluginList.js file.  The code will iterate through the installed list first so installed plugins show at the top of the list so they can be easily updated, then it will iterate through the list of plugins in the javascript array.  For each plugin, it will iterate through the list of compatible versions in the pluginInfo.json.  If a compatible version is found or the plugin is already installed (even if incompatible), the plugin will be listed in the top area of the page as a compatible plugin.  If there are no compatible versions of the plugin found, it will be listed at the bottom of the page in an 'incompatible plugins' area.  If the plugin is installed already, but the version installed is not compatible, then the UI will put a warning below the plugin info to indicate that it may be incompatible with the current FPP version.
- The install button is not shown for incompatible plugins.  Installed plugins can be upgraded (using git pull) or uninstalled.
- Each compatibility 'version' of a plugin has a branch and an optional sha1 hash.  The has is used if the author is doing development and only wants a specific version installed, but if empty, the plugin will be installed from the latest code in the branch.

The pluginInfo.json for a plugin looks like this:
Code: [Select]
{
    "repoName": "fpp-plugin-Template",
    "name": "Template Plugin for FPP Plugin developers",
    "author": "John Doe (jdoe)",
    "description": "This template plugin is designed to make it easier for plugin authors to create new FPP Plugins.  You can manually clone the git repository or install the Template Plugin and then go to the plugin's config page and use the form to copy the template into a new plugin directory.",
    "homeURL": "https://github.com/FalconChristmas/fpp-plugin-Template",
    "srcURL": "https://github.com/FalconChristmas/fpp-plugin-Template.git",
    "bugURL": "https://github.com/FalconChristmas/fpp-plugin-Template/issues",
    "versions": [
        {
            "minFPPVersion": 2.0,
            "maxFPPVersion": 0,
            "branch": "master",
            "sha": ""
        }
    ]
}

So, what am I missing? :)
-
Chris

Online Poporacer

  • Hero Member
  • *****
  • Join Date: Dec 2017
  • Location: Meridian Idaho
  • Posts: 642
  • Kudos: 9
Re: FPP v2.x plugin integration
« Reply #16 on: January 16, 2019, 07:34:18 AM »
I really like the idea of providing a centralized repository! Because you are iterating through the installed Plugins already, why not indicate on the page if there is an update available similar to what FPP does in the About page? Also, will there be a  time when a Plugin Requires a Script or other Plugin to function? If so, add that to the pluginInfo.json and if there are any Plugins installed that do not have the dependencies installed, flag that as well and indicate what Plugin/Scripts are missing with maybe a link to the required files? What about a Help or Installation URL as well for each Plugin?
I would love to help out testing and any other things that you will need to make this happen!
If to err is human, I am more human than most people.

Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #17 on: January 16, 2019, 12:10:00 PM »
I added a note about dependencies to my list and updated the pluginInfo.json file in the template plugin example.  I do plan on checking to see if there are any updates to the installed plugins and highlighting those.  Perhaps even an 'upgrade all' button.  I think that the home URL can point to wherever the plugin author wants, it could be the github repo or wiki page or somewhere else with other help pages, so I'm not sure about adding more URL's to the list.  I plan on adding support for the plugin to put a menu entry under any of the main menu items, so plugins can have their own entry under the Help menu.  The sample template plugin would have stub menu entries and .php files for each of these.

Offline Bshaver

  • Developer
  • ******
  • Join Date: Aug 2014
  • Location: Denver, CO
  • Posts: 1,426
  • Kudos: 22
Re: FPP v2.x plugin integration
« Reply #18 on: January 17, 2019, 09:30:22 AM »
is this in 2.x Master?
Denver Colorado
45,000 blinky blinks mix of pixels and 130 AC Channels (powered by Minleon & Lynx) ALL Powered by FPP and Falcon!

Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #19 on: January 17, 2019, 09:52:43 AM »
is this in 2.x Master?

Not yet, I hope to commit it tonight or tomorrow.  I have a few things I want to do before I push the new code to master:

- update the plugin install PHP code to use the new info rather than pluginData.inc.php.  This will still need a bit of work for dependencies.
- update my two plugins to create their pluginInfo.json file
- update other plugins under FalconChristmas org to create their pluginInfo.json files

If you want to go ahead and start creating pluginInfo.json files for your plugins, then I can add those links as well.  I think we can update links if/when we move plugins to the FalconChristmas org, I'm not as concerned about that right now as I am getting the structure in place.

Below is the format I have now.  I have added a place for plugin, script, and package dependencies.

Code: [Select]
{
    "repoName": "fpp-plugin-Template",
    "name": "Template Plugin for FPP Plugin developers",
    "author": "John Doe (jdoe)",
    "description": "This template plugin is designed to make it easier for plugin authors to create new FPP Plugins.  You can manually clone the git repository or install the Template Plugin and then go to the plugin's config page and use the form to copy the template into a new plugin directory.",
    "homeURL": "https://github.com/FalconChristmas/fpp-plugin-Template",
    "srcURL": "https://github.com/FalconChristmas/fpp-plugin-Template.git",
    "bugURL": "https://github.com/FalconChristmas/fpp-plugin-Template/issues",
    "versions": [
        {
            "minFPPVersion": 2.0,
            "maxFPPVersion": 0,
            "branch": "master",
            "sha": ""
        },
        {
            "minFPPVersion": 0,
            "maxFPPVersion": 1.11,
            "branch": "v1.x",
            "sha": ""
        }
    ],
    "dependencies": {
        "plugins": [
            "fpp-plugin-CoolPlugin1",
            "fpp-plugin-CoolPlugin2"
        ],
        "packages": [
            "system-package-name1",
            "system-package-name2"
        ],
        "scripts": [
            "Control/script-repository-script1",
            "PixelOverlay/script-repository-script2"
        ]
    }
}

Offline Bshaver

  • Developer
  • ******
  • Join Date: Aug 2014
  • Location: Denver, CO
  • Posts: 1,426
  • Kudos: 22
Re: FPP v2.x plugin integration
« Reply #20 on: January 17, 2019, 10:06:40 AM »
I will.
I'm not clear on the dependencies. If something is dependent on lets say . the Matrix tools plugin. What do I put in the dependencies section? The entire GIT URL? Or the common name?


For Example: the Git Repository for Matrix tools is: https://github.com/cpinkham/fpp-matrixtools
The common name in the Plugin installer is "Matrix Tools"
The folder name on the FPP instance is "fpp-matrixtools"


What if an author changes the folder name under "/home/fpp/media/plugins" then the folder name dependency will not be valid.


Also, the System package name: for example: i need sqlite3. Do I simply put sqlite3 in there as a dependency?


Also, the scripts, is that an installed script from the repository that is a dependency?
If you dont need any dependencies, do you leave the tree in there, but just be blank?


Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #21 on: January 17, 2019, 10:52:10 AM »
I'm not clear on the dependencies. If something is dependent on lets say . the Matrix tools plugin. What do I put in the dependencies section? The entire GIT URL? Or the common name?

In the dependency section, you would put the following:

- plugin repository name (instead of using a short name, we use the repository name, so 'fpp-matrixtools')
- Linux OS package name (ie, 'vorbis-tools' if you need the vorbis tools package installed)
- FPP Script repository name (ie, 'Control/StartEffectLooping.sh' if you need the StartEffectLooping script)

To make things simpler, we will always clone the git repo into a subdir that is the name of the repo, so instead of the 'short name' in pluginData.inc.php, we have a 'repoName'.  The folder name in /home/fpp/media/plugins will always be the name we use to refer to the plugin internally.  The one case this might be off is if someone makes a new plugin locally and the folder name disagrees with the repoName specified in the folders pluginInfo.json file.  But if that plugin were added to the list and installed later, it would be installed in a folder named using the repoName which is the git repository name.

If you don't need any scripts installed, then you can leave out the scripts section.  If you don't need any dependencies at all, you can leave out the whole dependencies section.  The 'versions' section is required though since it lists the branch to use.  If your package is simple enough to be compatible with all FPP versions, you can use 0 for the min and max versions.  If there is no minimum version, use 0.  If there is no maximum version, use 0.

Instead of pluginData.inc.php, we have fpp/js/pluginList.js which contains a JavaScript array definition which contains the list of available plugins.  The list contains two things.
 First is the repoName which would match the repoName in the plugin's pluginInfo.json file and second is a link to the plugin's pluginInfo.json file.  We wouldn't keep any other info about the plugin in the FPP git anymore, everything would be loaded from the plugin's pluginInfo.json in the plugin's git repository or from a locally installed plugin's pluginInfo.json file under /home/fpp/media/plugins/PLUGINRepoName/pluginInfo.json.  So that implies one more thing.  In the plugin's git repository, the pluginInfo.json file must be in the top level directory so that it is in the proper place once the repo is cloned under /home/fpp/media/plugins.  If the plugin author uses multiple branches then there would actually be multiple copies of pluginInfo.json.  The version inside a branch would really only need to know about that branch, but the main one linked to from FPP's pluginList.js would need to list all versions and branches.


Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #22 on: January 18, 2019, 04:20:04 PM »
I pushed the new code to the master branch.  Most details are in the commit log https://github.com/FalconChristmas/fpp/commit/4d8dd95, but the format of the pluginInfo.json file is in the fpp-plugin-Template repo at https://github.com/FalconChristmas/fpp-plugin-Template

To Do items still:

- Update installer to install dependency scripts/plugins/packages
- Add a 'Check for updates' button and enable updating plugins.  Rather than going out and checking 15-20 remote git repositories every time we load the page, I plan on making the 'check for updates' button run 'git fetch' in the local plugin directories to see if there are any updates.  This will use git to cache the updates locally and then the user could opt to upgrade a single plugin or all plugins.

I already added pluginInfo.json files for the fpp-vastfmt and big-green-button plugins which are in the FalconChristmas org.  In order for other plugins to start showing up, they will need pluginInfo.json files added to their repository and then a line added to fpp/js/pluginList.js to point to the pluginInfo.json file.

Let me know what you think or if you see anything that needs to be changed.

It doesn't have to be done now, but eventually I'd like to update the fpp-plugin-Template plugin to add logic to allow cloning the template into a new plugin.  The idea is that a user could make  new git repository, then go into the Template plugin's config page and enter the git repository URL and hit a button to 'clone' the template plugin.  The clone script would "git clone" the user's empty git repository into /home/fpp/media/plugins then copy the required files such as the pluginInfo.json, menu.inc, scripts, etc. from the template into the new plugin directory.  The 'clone' form could prompt for a plugin name, author, description, etc. and put those into the new pluginInfo.json file.

Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #23 on: January 18, 2019, 10:29:04 PM »
One more change.

I'm thinking now that it would be best if the dependencies were under the versions list.  Since some versions might not have the same dependencies as others.

here's the current copy in fpp-plugin-Template:

Code: [Select]
{
    "repoName": "fpp-plugin-Template",
    "name": "Template Plugin for FPP Plugin developers",
    "author": "John Doe (jdoe)",
    "description": "This template plugin is designed to make it easier for plugin authors to create new FPP Plugins.  You can manually clone the git repository or install the Template Plugin and then go to the plugin's config page and use the form to copy the template into a new plugin directory.",
    "homeURL": "https://github.com/FalconChristmas/fpp-plugin-Template",
    "srcURL": "https://github.com/FalconChristmas/fpp-plugin-Template.git",
    "bugURL": "https://github.com/FalconChristmas/fpp-plugin-Template/issues",
    "versions": [
        {
            "minFPPVersion": 2.0,
            "maxFPPVersion": 0,
            "branch": "master",
            "allowUpdates": 1,
            "sha": "",
            "dependencies": {
                "plugins": [
                    "fpp-plugin-CoolPlugin1",
                    "fpp-plugin-CoolPlugin2"
                ],
                "packages": [
                    "system-package-name1",
                    "system-package-name2"
                ],
                "scripts": [
                    "Control/script-repository-script1",
                    "PixelOverlay/script-repository-script2"
                ]
            }
        }
    ]
}

Offline jcross

  • Developer
  • ******
  • Join Date: Dec 2014
  • Location: Brandon, Manitoba, Canada
  • Posts: 141
  • Kudos: 1
    • Crosslights Christmas
Re: FPP v2.x plugin integration
« Reply #24 on: January 19, 2019, 01:28:25 AM »
One more change.

I'm thinking now that it would be best if the dependencies were under the versions list.  Since some versions might not have the same dependencies as others.

here's the current copy in fpp-plugin-Template:

Code: [Select]
{
    "repoName": "fpp-plugin-Template",
    "name": "Template Plugin for FPP Plugin developers",
    "author": "John Doe (jdoe)",
    "description": "This template plugin is designed to make it easier for plugin authors to create new FPP Plugins.  You can manually clone the git repository or install the Template Plugin and then go to the plugin's config page and use the form to copy the template into a new plugin directory.",
    "homeURL": "https://github.com/FalconChristmas/fpp-plugin-Template",
    "srcURL": "https://github.com/FalconChristmas/fpp-plugin-Template.git",
    "bugURL": "https://github.com/FalconChristmas/fpp-plugin-Template/issues",
    "versions": [
        {
            "minFPPVersion": 2.0,
            "maxFPPVersion": 0,
            "branch": "master",
            "allowUpdates": 1,
            "sha": "",
            "dependencies": {
                "plugins": [
                    "fpp-plugin-CoolPlugin1",
                    "fpp-plugin-CoolPlugin2"
                ],
                "packages": [
                    "system-package-name1",
                    "system-package-name2"
                ],
                "scripts": [
                    "Control/script-repository-script1",
                    "PixelOverlay/script-repository-script2"
                ]
            }
        }
    ]
}


I was thinking about a few items but you captured most of them in recent updates and others (like the ability for different repos for different hardware (pi/bbb,etc)) likely wouldn't be a good idea as differences likely should be handled in the plugin code.


I have added the pluginInfo.json file to the fpp-after-hours plugin repo.


I'm interested in seeing how the package dependencies part works out.
When a plugin is uninstalled will fpp eventually remove dependencies that are no longer required by any plugins?


Great work on this! I'm really happy to see this work going into plugin management.


Update: Just updated my dev box and the Plugin menu item is no longer clickable :)

Offline Bshaver

  • Developer
  • ******
  • Join Date: Aug 2014
  • Location: Denver, CO
  • Posts: 1,426
  • Kudos: 22
Re: FPP v2.x plugin integration
« Reply #25 on: January 19, 2019, 09:45:06 AM »
Chris:


This line: [/size]In order for other plugins to start showing up, they will need pluginInfo.json files added to their repository and then a line added to fpp/js/pluginList.js to point to the pluginInfo.json file.


This is expecting that we have to have an admin check in the line to the plugin repository. The idea to have the GIT URL from a repository that the user wants to add put it in there, and click "ADD" or something like that would do this action for them.  Is that in the works still. This would not require any user to wait for one of the admins that know how to check in the plugin code to get new features...


Offline Bshaver

  • Developer
  • ******
  • Join Date: Aug 2014
  • Location: Denver, CO
  • Posts: 1,426
  • Kudos: 22
Re: FPP v2.x plugin integration
« Reply #26 on: January 19, 2019, 09:47:16 AM »
Sorry the font size... the line in your post that says the url to the pluign has to be added to the .js file.  sorry i could not post.

Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #27 on: January 19, 2019, 10:16:11 AM »
I have added the pluginInfo.json file to the fpp-after-hours plugin repo.

I'm interested in seeing how the package dependencies part works out.
When a plugin is uninstalled will fpp eventually remove dependencies that are no longer required by any plugins?

Update: Just updated my dev box and the Plugin menu item is no longer clickable :)

I haven't thought about cleaning up package dependencies.  I don't know that that is a huge deal unless a package says it wants to install libreoffice or something huge, and even then it's not that important given the size of the microSD.

I added your plugin to js/pluginList.js, but this reminded me of another ToDo.  I need to update the version compare logic to understand that 2.1150 is greater than 2.3.  I ran into this while testing something with a plugin the other day and forgot to write it down.  I'll try to get that fixed tonight.  For now, that means that your plugin shows up in the incompatible section. :(

For the menu, there is no 'plugin' menu item anymore.  The existing item was renamed to 'Plugin Manager' and there is the a 'Plugins' divider in the menu between the regular FPP menu entries at the top of the menu and the plugin menu entries at the bottom of the menu.  I just pushed a change to make this divider stand out a little more.  If you guys think we don't need it, I can remove it totally, but I was thinking that it might make it clearer which were plugins and which weren't.

Offline CaptainMurdoch

  • Administrator
  • *****
  • Join Date: Sep 2013
  • Location: Washington
  • Posts: 9,856
  • Kudos: 214
Re: FPP v2.x plugin integration
« Reply #28 on: January 19, 2019, 10:23:24 AM »
This line: In order for other plugins to start showing up, they will need pluginInfo.json files added to their repository and then a line added to fpp/js/pluginList.js to point to the pluginInfo.json file.

This is expecting that we have to have an admin check in the line to the plugin repository. The idea to have the GIT URL from a repository that the user wants to add put it in there, and click "ADD" or something like that would do this action for them.  Is that in the works still. This would not require any user to wait for one of the admins that know how to check in the plugin code to get new features...

Yes, basically we need to know where to search for plugins at, so there still has to be something in the FPP repository unless we move the list of plugins into a shared repository that plugin authors can each edit.  If we do that, we could retrieve the list from the shared repository.   The big difference between now and previous is that the plugin author can do pretty much anything to an existing plugin, the only thing they can't do is add a new plugin to the list that everyone sees.  I'm totally fine with moving this file to a different git repository and then granting access to that repository to make it easier for you to add new plugins to the list.  Do you think we should do that?

Once a plugin is in the list, updates don't require any changes on the FPP git side.  The plugin's pluginInfo.json file has the info about upgrades and other versions of the plugin and that pluginInfo.json file is in the plugin's repository, not the FPP repository.  This puts the plugin author in control of versions, description, changing URL's, changing the SHA for the default install version, etc., none of those require a change in the FPP repo.

The new plugin manager/installer page has a section right at the top to allow a user to install a plugin which is not in the official list in the FPP repository.  Attached is a screenshot.

Offline Bshaver

  • Developer
  • ******
  • Join Date: Aug 2014
  • Location: Denver, CO
  • Posts: 1,426
  • Kudos: 22
Re: FPP v2.x plugin integration
« Reply #29 on: January 19, 2019, 10:38:45 AM »
Ok - Thanks. That achieves what I was looking for. Thank you. This would allow an author to TEST the install capabilities of a plugin as well without the publishing to the main Plugin list. 


Agreed with a note somewhere that "If you do not see the plugin that you are looking for, you can install it manually "HERE...".


I dont want to break something by editing a file and checking it in and breaking something. I did that once... :( . But the ability to have something listed, or moderated using an ISSUES list on GitHub, where the url to the pluginInfo.json file can be posted there and those that are experienced, can add it to the REPO file.  the plugin install is Use at your own risk if it is not in the Main Repo list.


I have a good framework to continue. and I like the ability to clone the framework for new people.


 

Back to top