/*******************************************************************************

    Author ......... The Cacti Group
    Contact ........ cacti-user@lists.sourceforge.net
    Home Site ...... http://www.cacti.net
    Program ........ Cacti Performance Booster
    Version ........ 5.0
    Purpose ........ Large Site Performance Booster for Cacti

*******************************************************************************/


----[ Purpose

    This plugin boost's Cacti performance especially for Large Sites.  It does
    this by introducing three new features to Cacti.

    First, it caches recently viewed images to a public Cache folder for all
    users to share.

    Second, it introduces an "on demand" RRD update feaure to Cacti.  This
    feature will only update the RRD files when there is demand on the system
    to view a graph.  The RRD's will be updated just before the Graph is rendered
    by the web server.  Then, on a predetermined schedule, it conducts a batch
    update's of all remaining RRD's.

    Lastly, it introduces an RRD update service to Cacti.  This service allows
    you to add multiple Cacti servers to your web farm allowing all the servers
    to participate in the "on demand" RRD update and viewing process.

----[ Features

    On Demand RRD Updates
    Timed and Mass RRD Updates by Number of Records
    RRD Update Service
    PNG Caching

----[ Prerequisites

    Before you install Boost, you need to have met the following pre-requisites:

    1) Cacti 0.8.7 and above.
    2) Cacti Plugin Architecture v2.x
    3) MySQL 5.0 or above.  Versions of MySQL less than 5.0 are not supported.
    4) Have quite a bit of system memory available to hold your memory resident
       database and/or your sql results during major rrd update cycles.
    5) Switching the effective userid to "cactiuser" requires posix support.
       Many installations come with php posix support enabled. Other may require
       additional packages (e.g. php-process for RH based systems)

    Before enabling boost, please consider very carefully how you plan to use it
    long term.  Carefull thought should be given to how often you update RRD's
    at what point you need to either increase your MySQL max heap table size,
    or increase your update frequency.  You should also consider carefully how
    much memory to allocate to PHP when retrieving records during major updates.

    If you are unsure of what I am talking about, you should not use Boost!!!

----[ Installation

    Install is similar to other plugins with the following notes:

    0) Check permissions, the tar file was made on Windows.  My next PC will
       be Linux. :)

    1) You must first choose your RRD update cache database format.  There are
       two options:

       - Memory Resident (MySQL 5.x and Above)
       - MyISAM (All Supported MySQL Versions)

       The benefit of the Memory Resident impmenentation is that database
       exchanges will be very fast and will not delay the system.  However,
       if your system is rebooted between major update cycles, you can loose
       some graph data.

       The MyISAM version will increase MySQL load during polling, but does
       store the information in a lossless (well almost anyway) format.

       IMPORTANT: If you plan on using a memory resident database format,
       make sure you set the mysql environment variable "max_heap_table_size"
       large enough to accomodate all of your records, else, you will loose
       data.  This environment must go into either your my.cnf or my.ini file.

       To install either format, you must follow the instructions below:

       mysql -u <user> -p cacti < "my_chosen_format.sql"

       Options here are as follows:

       boost_sql_memory.sql, boost_sql_myisam.sql

       NOTE: You may skip this step.  If you do, the tables will be initialized
       as memory tables.

       Note: if you choose the memory table option, you may want to limit the
       size of the output field to your maximum output length from all polled
       items as MySQL varchar(512) take 512 bytes of memory even if your output
       is simply a 'U'.
       
    2) Once you have the SQL in place, you must enable you plugin by going to
       the Plugin Management interface and installing and enabling the plugin.
       Please note, that if you are running a prior version of Boost, it might
       be a good idea to clear out the boost cache before doing any of this.

    3) Then, you must decide on a directory to hold your image cache, and verify
       permissions both their and in the rra directory.  If you are not using
       the boost server, the web server account will need R/W into both directories
       and be able to create files in those directories as well.

    4) Next you have to enable any of the three features mentioned.  You can
       do this by going to your Cacti website, logging in as an adminnistrator
       and going to the Settings->Boost tab.

    6) If your Web server is not going to have access to create/update RRD
       files, you must start the "boost_server.php" file as a service either
       as an init.d service, or more reliably as an inittab service.

       NOTE: The Inittab Process has the benefit of being able to restart the
       boost server if it crashes and therefore is preferred. Same holds for
       the new "upstart" procedure.

       In the UNIX/LINUX environment, to use the init.d process, use a script
       attached cacti_rrdsvc script and follow the instructions below.

       ------------------------------------------------------------------------
       Init.d Process:
       ------------------------------------------------------------------------

       a) Place the cacti_rrdsvc in the /etc/init.d folder.

       b) Edit cacti_rrdsvc and modify the variable PROGPATH to poing to the
          location of boost_server.php, and the DEBUBLOG entry.

       c) Edit both boost_server.php and boost_rrdupdate.php and verify the PHP path
          in the first line of the file.

       d) Mark the boost_server.php and boost_rrdupdate.php files as executable:

          chmod +x boost_server.php
          chmod +x boost_rrdupdate.php

       e) If using an id other than root to update the rrdfiles mark the boost_server.php
          and boost_rrdupdate.php files setiud:

          chown root:root boost_server.php
          chmod +s boost_server.php
          chown root:root boost_rrdupdate.php
          chmod +s boost_rrdupdate.php

       f) Edit the boost_server.php and make sure the embedded path to PHP is
          correct.

       g) Run the following commands.

          On Most Linux Variants:
          dos2unix cacti_rrdsvc (sorry, I'm still writing in Windows)
          chkconfig --add cacti_rrdsvc
          chkconfig cacti_rrdsvc on

       h) Then either:

          service cacti_rrdsvc start [debug]

          or:

          /etc/init.d/cacti_rrdsvc start [debug]

          The "debug" option is optional

       i) Make sure it's running by issuing the following command:

          ps -ef | grep cacti_rrdsvc

       ------------------------------------------------------------------------
       Inittab Process:
       ------------------------------------------------------------------------

       NOTE: this process varies by Operating system.  For this example, we will
       assume a Linux install.  On Windows, you should create a service.  I will
       not be covering how to do that here.

       a) Edit /etc/inittab and Place a new entry in the file as such:

          # Cacti Boost Server
          bsts:345:respawn:/var/www/html/cacti/plugins/boost/boost_server.php

          You may replace bsts with the unique ID of your choice.

       b) Edit both boost_server.php and boost_rrdupdate.php and verify the PHP path
          in the first line of the file.

       c) Mark the boost_server.php and boost_rrdupdate.php files as executable:

          chmod +x boost_server.php
          chmod +x boost_rrdupdate.php

       d) If using an id other than root to update the rrdfiles mark the boost_server.php
          and boost_rrdupdate.php files setiud:

          chown root:root boost_server.php
          chmod +s boost_server.php
          chown root:root boost_rrdupdate.php
          chmod +s boost_rrdupdate.php

       e) Refresh/Reload the inittab using the following command:

          init q

       f) Make sure it's running by issuing the following command:

          ps -ef | grep boost_server.php

       ------------------------------------------------------------------------
       Upstart Process:
       ------------------------------------------------------------------------

       NOTE: this process varies by Operating system.  For this example, we will
       assume a Linux install.  On Windows, you should create a service.  I will
       not be covering how to do that here.

       a) Copy cacti_boost.conf to /etc/init.
          Edit that file and enter the paths for boost_server.php and the log file

       b) Edit both boost_server.php and boost_rrdupdate.php and verify the PHP path
          in the first line of the file.

       c) Mark the boost_server.php and boost_rrdupdate.php files as executable:

          chmod +x boost_server.php
          chmod +x boost_rrdupdate.php

       d) If using an id other than root to update the rrdfiles mark the boost_server.php
          and boost_rrdupdate.php files setiud:

          chown root:root boost_server.php
          chmod +s boost_server.php
          chown root:root boost_rrdupdate.php
          chmod +s boost_rrdupdate.php

       e) Start the boost server using:

          initctl start cacti_boost

       f) Make sure it's running by issuing the following command:

          ps -ef | grep boost_server.php

    7) Finally, you can poll the server using telnet.  To do so, start a telnet
       session on the port you choose to use (default 9050) and type the
       "status" command followd by a carriage return.  You should receive a
       favorable response.

----[ Additional Help?

    If you need additional help, please goto forums.cacti.net.

----[ Possible Bugs?

    If you figure out this problem, let me know!!!

----[ Special Thanks

    Ernest Wilson (N3NCY) - For allowing me to break and then fix his
    system repeatedly perfecting this Plugin.

    Jimmy Connor (cigamit) - For bringing the plugin architecture to
    the world of Cacti and provding continual support of my development
    efforts.

    dononeil (dononeil) - For his help with the inittab process setup
    documentation for Linux.

    Boris Lytochkin (BorisL) - For his idea of using a lock file to detect
    process overruns correctly and inspiring me to complete Boost 2.0.

----[ Future Changes

    Got any ideas or complaints, please e-mail me!

----[ Changelog
   --- 5.1 ---
   feature: support for ifHighSpeed

   --- 5.0 ---
   feature: Another one deep performance tuning and code cleanup
   feature: add upstart file to start boost_server
   bug: replace some deprecated statements
   bug: cope with E_DEPRECATED
   compat: adding some more logging to cope with setuid issues
   
   --- 4.3 ---
   bug: Correct misspelling
   bug: Plugin Hook Registration for Fetch on Upgrade is not enabled by default
   feature: Set the permissions on the Cache files so that they can be managed by the poller
   feature#0001853: Boost v4.2 does not check graph size before pushing cached image 

   --- 4.2 ---
   compat: Don't override Cacti log level
   bug: eliminate harmless temporary table errors
   bug: when disabled, this plugin should not run
   bug: warn if you are attempting to use boost redirect and spine isn't in use

   --- 4.1 ---
   bug: Major issue with BTREE PRIMARY KEYS and MEMORY TABLES in MySQL causing gaps
   feature: When boost redirect is not in use, do not update to the second to prevent graph breakage

   --- 4.0 ---
   feature: Add support for Cacti's new rrdtool segfault detection
   bug: Setting logging to MEDIUM and higher causes boost images to break

    --- 3.1 ---
    bug: Boost poller statistics logging missing global variable declaration
    bug#0001643: When boost 3.0 plugin is installed and then uninstalled it doesn't cleans database entries properly.
    bug#0001555: Boost errors generated in log when another user has graphs open 
    feature: Allow boost to recover from rrdtool crashes
    compat: Make the error handler not require PHP 5.2
    compat: Remove PIA 1.x support

    --- 3.0 ---
    feature: Deep performance tuning and code cleanup
    feature: Add logging to boost updates for performance tuning
    feature: Add hooks to support 95th Percentile and Bandwidth Summation

    --- 2.6 ---
    bug: Initial RRDfile update taks two boost cycles and not one
    bug: Cacti RRDtool fetch command bypasses boost causing issues
    bug: Don't optimize boost table if it's in MEMORY storage engine

    --- 2.5 ---
    feature: Integration with DSStats
    feature: Support boost redirect for large systems
    feature: Properly support structured paths

    --- 2.4 ---
    bug: [Boost 2.3] Include line referring to thold plugin
    bug: Should be db_fetch_row and not db_fetch_cell
    feature: Allow specification of memory limits

    --- 2.3 ---
    feature: Better max memory reporting for older PHP installs
    feature: Reduce the load on the database server when displaying boost stats
    feature: Integrate better with tholds vrule display (image caching off)
    feature: Upgrade Automatically when either viewing the console or the plugin
             management page.

    --- 2.2 ---
    bug: If running the boost server, mysql can timeout causing 2006 errors
    bug: Typo relative to Linux Kernel versions in relative to command length
    bug: Logging did not work on all occasions due to an undefined variable error
    bug: Improve the counting algorythm to remove some bogus RRDtool update errors
    bug: Check the validity of Multivalue updates prior to allowing updates
    bug: Correct documentation relative to the the inittab process so that it works
    bug: Prevent more than one Boost Server from running at a time by validating
         Socket bind

    --- 2.1 ---
    bug: Correcting overrun issues
    bug: Rename maximum rows to maximum data sources
    bug: RRDupdate batch process was not working as designed causing excessive
         memory to be consumed in some cases
    feature: Log memory performance and make available via UI

    --- 2.0 ---
    bug: remove the base start time, it complicates things
    bug: make version, force, and debug options consistent
    bug: rrdupdates oftentimes caused gaps in graphs due to limit issues
    feature: add an rrdtool output logging option
    feature: support inittab for restarting the boost server
    feature: support process locking control methodology

    --- 1.8 ---
    bug: correct issue with multiple temporary tables using the same name are encountered
    bug: significant bug from 1.6 that prevents the poller_output_boost table from emptying
    completely


    --- 1.7 ---
    feature: Add Cacti 0.8.7 compatibility

    --- 1.6 ---
    feature: make the delete process a high performance one by batching the deletes
    bug: set a good memory limit and runtime in poller_boost.php

    --- 1.5 ---
    bug: Under some circumstanced the poller_boost.php could not be fored to run
    bug: Scripts that do not return the correct number of DS' did not update properly
    bug: New forcerun option introduced in version 1.3 was causing boost not disable properly
         when you unselected it from the user interface.

    --- 1.4 ---
    bug: Fixed an issue with data queries where the rrdupdate template was not formated correctly

    --- 1.3 ---
    bug: Fixed an issue where multi part responses with a "0" would return "U"
    feature: Added better logging functions to help in debugging
    feature: Allow you to increase the MySQL Insert string length to improve performance
    feature: Allow you to increase the RRDtool update string length to improve performance

    --- 1.2 ---
    bug: Added boost_server.php and poller_boost.php to the no session array
    bug: Made slight change to rrd-update functions to accomodate for abarant
    output in the poller_output_boost table.  This would cause the boost
    plugin to loose data if improperly formatted data made it into the table.

    --- 1.1 ---
    bug: Fix issues with Multiprocess RRDupdating of RRD Files

    --- 1.0 ---
    Initial release

    --- Known Issues ---
    None reported
