Cumulus MX FAQ: Difference between revisions
m (corrected a link) |
|||
(54 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
= Introduction = |
= Introduction = |
||
[[Category:Cumulus_MX]] |
|||
<div style="background: LemonChiffon;padding:5px; margin:2px;"> |
|||
[[File:Crystal Clear info.png|40px]] This document is 'Work In Progress' so content may not be complete or accurate! |
|||
</div> |
|||
If you are seeking information on why you might want to use Cumulus, then please read [[About Cumulus|the About Cumulus article]] first. That is a general introduction to what Cumulus does and what might be its advantages over alternative software. |
|||
It is anticipated that the main documentation for Cumulus MX will be placed in [[Cumulus MX|this article]] and you should read that to get basic information about MX. |
|||
This article is to answer those awkward questions that will arise when you try to go beyond the "normal" installation or you need some trouble shooting style help. The questions have been placed into groups, but this is my first attempt at designing this article, so please feel free to make improvements that suit the way you search. |
|||
{{TOCright}} |
|||
== Request for assistance == |
|||
If you, either need help on topics not covered in this article, or you see any messages that are not explained below, please update this article (you may need to ask for a login) or else post on the support forum in [https://cumulus.hosiene.co.uk/viewforum.php?f=38 Cumulus Wiki Suggestions for updates and additions] sub-forum. Thank you. |
|||
= Background = |
|||
Mark Crossley created this page on 26 October 2018 to help people with questions about MX where it works differently to Cumulus 1 and the [[FAQ|Cumulus 1 FAQ]] did not help. |
Mark Crossley created this page on 26 October 2018 to help people with questions about MX where it works differently to Cumulus 1 and the [[FAQ|Cumulus 1 FAQ]] did not help. |
||
Line 7: | Line 26: | ||
I started pasting in useful material, mostly from Steve Loft's posts in the forum about the beta version of MX, but as at that time I was not using MX I hoped somebody else would make sense of it and create a better layout. Of course nobody did. |
I started pasting in useful material, mostly from Steve Loft's posts in the forum about the beta version of MX, but as at that time I was not using MX I hoped somebody else would make sense of it and create a better layout. Of course nobody did. |
||
When I did start using MX, I decided to edit other articles first, see [[Cumulus MX]], [[Cumulus MX formal release versions]], [[Webtags]], [[Cumulus.ini]], [[Dayfile.txt]], [[Category:Log Files]], [[Php webtags]], and various others. |
When I did start using MX, I decided to edit other articles first, see [[Cumulus MX]], [[Cumulus MX formal release versions]], [[Webtags]], [[Cumulus.ini]], [[Dayfile.txt]], [[:Category:Log Files]], [[Php webtags]], and various others. |
||
So that left this page, |
So that left this page, but I don't have enough time to complete updating this article. If you read this, and believe you can help with completing this update, please do. |
||
= Installation and Upgrading problems = |
|||
== Basic advice == |
|||
# Whether you are installing MX for the first time, or updating to a new version, you download the zipped software from [[Software|Software page]]. |
|||
#Unzip it either directly to the place where you want it, or to a holding area where you keep downloaded software, and then use copy or another file transfer process to get it to final destination |
|||
#You will find more advice in the [[Cumulus_MX|Cumulus_MX documentation page]]. |
|||
#If there is a problem, the most likely problem is that you are trying to install onto say a SD card and that has failed, and needs to be replaced. You might want to consider investing in a SSD drive, they tend to be very reliable. |
|||
== I use Cumulus 1 and want to move to Cumulus MX == |
|||
Please see [[Moving from Cumulus 1 to MX]] article. |
|||
== Variation between releases == |
|||
*Cumulus MX is currently being developed |
|||
*#Some functionality changes between version |
|||
*Some bugs probably remain from the older beta versions of MX initiated by Steve Loft. |
|||
*Each new piece of functionality has potential to add further bugs as it is impossible for the developer to test that feature with all possible weather station types, and with all setting permutations |
|||
* Most releases are tested by a few beta testers prior to them being released generally, but again these people don't test all permutations. |
|||
*#Consequently, if you always upgrade to the latest version, you might encounter new functionality that possibly might not work with your particular set-up. |
|||
*#Equally, if you don't always update to the latest version, you might miss out on a solution to a problem in the version you are using |
|||
*The advice is therefore to retain the download of the version you are currently using when you upgrade to a new version, so regression is easy if you do encounter a problem with new functionality. |
|||
*Always read the release notice when a new release is available, that might determine the speed at which you should upgrade. |
|||
If you ever do need to download the distribution for a release prior to the current one, please go to [https://github.com/cumulusmx/CumulusMX/releases this web page]. Don't forget to read the release notices on the forum, because there might be differences between releases that require action by you (whether increasing version number or decreasing version number). |
|||
= Troubleshooting - messages in MXdiags or in console/terminal output = |
= Troubleshooting - messages in MXdiags or in console/terminal output = |
||
Mark Crossley lists the issues outstanding in beta version of MX [[MX Issues|in MX Issues article]]. Although that has not been updated since 2018, it might give you a hint of bugs that might still exist. |
|||
== Request for assistance == |
|||
=='''MESSAGE FROM SFWS'''== |
|||
If you see any messages that are not explained below, please update this article (you may need to ask for a login) or else post on the support forum in [https://cumulus.hosiene.co.uk/viewforum.php?f=38 Cumulus Wiki Suggestions for updates and additions] sub-forum. Thank you. |
|||
I do not have a separate environment to test MX in, and I don't want to mess up my set of weather measurements by any action causing MX to give an error message just to be able to get this article right. |
|||
== Questions == |
|||
Also I don't want to search through all the source code to look up every possible message that MX can produce, I know different versions of MX can differ in what messages they output to the diags, so it is not simple to know what to include here. It may be I have matched the message MX gives to the wrong solution because I do not have a complete knowledge of how MX works. Please do correct any mistakes I have made and improve this article, I cannot do it all alone. |
|||
So forgive me if actual MX messages are different to those I quote here, I just hope that you find what I have typed here helpful. |
|||
If you see another message, please add the message and solution here, so it becomes a better place to help people solve any problems they encounter. |
|||
== Questions and answers == |
|||
=== Not watching the terminal or console window === |
|||
Although the MX engine writes messages to the terminal or console (the terminology varies depending on the device) many people don't have a monitor connected to the device running MX (and if they do, they don't watch this output). Therefore it is easy to not to see messages, and not to know when MX is running, miss out when it is reporting errors, or perhaps even to fail to spot when it crashes. |
|||
One problem people have is stopping MX while it is busy, perhaps still doing catch-up from your station's logger after restarting, perhaps doing an end of day or end of month rollover, maybe in the middle of some other task. It is not so easy to see whether MX is running, nor when it is busy. MX does so many tasks now, that picking a moment when it is not doing something is harder. |
|||
Although MX is designed assuming it is left running all the time, it is permissible to stop, and restart it, just do that as few times as possible (once a day if you must). MX will run on a variety of devices, so can run on devices that are cheap to leave on all the time. |
|||
=== Station type not known === |
|||
When you first run MX, it needs to be told what station type to look for, and how to connect to that station. Start up the [[Cumulus_MX#Administrative_Interface|Administrative Interface]] and select the settings tab, it should then display station settings for you to make the necessary choices. |
|||
=== Unable to connect to station === |
|||
This means that you have not set all the settings mentioned in previous answer. |
|||
*Maybe you have selected a station type where you need to select a connection port |
|||
*Maybe you have migrated from Cumulus 1 to MX. These two flavours use a different setting to select the port, so MX ignores the C1 setting ('''Port=number''' in [[Cumulus.ini#Station|[Station] section of Cumulus.ini configuration file]]) and you need to enter the one MX uses ('''Comport=text''' in Cumulus.ini). |
|||
=== Weather Station Problems === |
|||
If one of the sensors on your weather station fails, MX will reuse the same value for 6 reads, after that it will conclude the station is fault, and with default settings, MX will stop working. |
|||
There are many places where you can look on the web for advice on setting up a weather station. In the UK, the best site to read first is '''the resources linked from''' [https://www.rmets.org/resource/how-can-you-make-high-quality-weather-observations how-can-you-make-high-quality-weather-observations] page. |
|||
=== The Moon rise or set time is showing dashes. === |
|||
The moon doesn’t rise and set every day, the dashes appear when that event doesn’t occur on that particular day. The period between a moon rise and set is approximately 12 hours and 25 minutes, on average. The moon rises (or sets) approximately 50 minutes later, on average, each day. It therefore follows that sometimes the moon will not rise or set on a particular day - i.e. it had risen the previous day, or will set on the following day. |
|||
=== The Moon image is just black === |
|||
It's probably a New Moon. |
|||
=== I have a message about web tags === |
=== I have a message about web tags === |
||
There is an article about [[Webtags|Web tags]], I have tried to keep that up to date. That may help, the most common errors seem to be: |
There is an article about [[Webtags|Web tags]], I have tried to keep that up to date. That may help, the most common errors seem to be: |
||
# Using a web tag not available at the version you are using (the message in the |
# Using a web tag not available at the version you are using (the message in the MXDiags should indicate which one). |
||
# Using the wrong output formatters, as the article referenced above explains, the codes to use for MX are far more complicated. |
# Using the wrong output formatters, as the article referenced above explains, the codes to use for MX are far more complicated. |
||
# Using an input modifier that MX cannot match to data available (the message will talk about arrays). |
# Using an input modifier that MX cannot match to data available (the message will talk about arrays). |
||
Line 29: | Line 117: | ||
This could cover a lot of problems. Make sure you don't have 2 copies of MX running and are not running anything else that accesses the same weather station or the same files as MX, because this message may relate to such a conflict. |
This could cover a lot of problems. Make sure you don't have 2 copies of MX running and are not running anything else that accesses the same weather station or the same files as MX, because this message may relate to such a conflict. |
||
Maybe there is some fault with your weather station to MX communication. |
|||
Other problems that crash MX when you have just updated either MX or Mono, may relate to code added in update. For example 3.6.2 crashed in some contexts because of a new "FluentFTP.dll file", 3.6.3 reverted to the one used for 3.6.0. Other people have seen problems because of changes in how MONO worked on an update, although that was more of a problem with MX beta than later MX. |
Other problems that crash MX when you have just updated either MX or Mono, may relate to code added in update. For example 3.6.2 crashed in some contexts because of a new "FluentFTP.dll file", 3.6.3 reverted to the one used for 3.6.0. Other people have seen problems because of changes in how MONO worked on an update, although that was more of a problem with MX beta than later MX. |
||
Maybe MX is trying to upload to your web server and that web server is not currently available. Or MX is trying to upload to a database server, and that is not currently available. |
|||
=== I have a message about missing web templates === |
|||
If you have selected the '''Include standard files''' option, MX expects to find all the standard web templates in the folder ''CumulusMX/web'', and will give an error message if it cannot find that folder or cannot find all the files in that folder that are provided in each MX distribution. |
|||
= Accessing the Admin Interface = |
|||
The admin interface is where you set all settings for MX. It also provides screens for editing log files and the extremes they contain. Finally it provides a number of pages where you can view the weather data Cumulus MX is processing. |
|||
== Questions == |
|||
Most problems will be in gaining access. |
|||
# Read [[Cumulus_MX#Administrative_Interface|the documentation first]] to see how to access this interface. |
|||
#Because the MX engine generates a web server on which to run the interface, there might be a permissions problem |
|||
#*You might need to issue a one-off command to let the MX engine generate the web server, this often involves giving it '''root''' or '''administrator''' privileges although it does not need to run at the root or run in an administrator user. The command needed depends on your operating system, see [[Cumulus_MX#Installing_and_Running_Cumulus_MX|the documentation]], to see if it is covered there. |
|||
#*You may need to edit settings in your virus checker, firewall, or similar software to open the '''8998''' port (or if you need to use a different port, start MX with the relevant port parameter). |
|||
= Problems with MX not accepting data when move from C1 = |
= Problems with MX not accepting data when move from C1 = |
||
Line 42: | Line 150: | ||
=== Ignoring earlier data === |
=== Ignoring earlier data === |
||
There are two contexts where new Cumulus MX users have reported problems stopping them seeing some earlier data they saw when they were using Cumulus 1 (C1). |
|||
Two issues here. |
|||
# MX needs to read from [[Cumulus.ini]] the earliest date to process as it ignores any content for earlier dates. (Cumulus 1 just read all the files it could find in the data folder. |
# MX needs to read from [[Cumulus.ini]] the earliest date to process as it ignores any content for earlier dates. (Cumulus 1 just read all the files it could find in the data folder. |
||
# When you first start running MX, it does not seem to be able to populate charts and recent history web tags for earlier times. |
# When you first start running MX, it does not seem to be able to populate charts and recent history web tags for earlier times. |
||
= Dealing with rogue values = |
|||
==INPUT ISSUES== |
|||
Input issues are when Cumulus reads the wrong information from the weather station. |
|||
* Sometimes a weather station will supply incorrect data bits that mean wrong numbers are input to Cumulus MX or that MX looking at byte level does not recognise what it is being sent |
|||
*What Cumulus outputs, for up to 3 different wind speeds it can calculate, depends on the weather station and which outputs from the weather station are processed by Cumulus. |
|||
*For Fine Offset (not Ecowitt), MX has various options on '''Settings''' menu, ''Calibration settings'' page, '''Spike removal''' section to reject any readings that ''differ from previous value'' by more than figure selected. As mentioned, this is not available for most weather stations, only those that can optionally output to Easyweather software. |
|||
*For all weather stations, MX has various options on '''Settings''' menu, ''Calibration settings'' page, '''offsets''' and '''multiplier''' sections to calibrate what is read from the weather station by applying a multiplier (in some cases both first and second order multipliers), and/or an offset. |
|||
*MX can also reuse the last good reading (for up to 6 times) if there is a problem with individual readings. After that it gives up and labels that sensor as not working (see next point). |
|||
*MX can stop running if it is unable to obtain readings for key measurements, i.e. temperature, pressure, wind speed, humidity, although there are some settings that can over-ride this action. |
|||
== DERIVATIVES ISSUES == |
|||
Derivatives are any values that Cumulus reports which it has not read directly from the weather station. In a sense, every out put by Cumulus is a derivative, because the units that the weather station supplies readings in, have to be converted to the units that the user has requested. |
|||
*Cumulus MX ignores any maximum and minimum values supplied by a weather station; it calculates these highest and lowest derived values each metrological day for itself, from every measurement it processes, not the small sample of it actually writes to the standard log. |
|||
**As each new standard sensor reading is read in it is first converted to units selected by user, then the reading is added to the recent history array, then it is compared against the previous highest/lowest for that meteorological day, updating those as needed, and finally the reading is placed into web tags (for web pages), application programming interface that can feed it to admin interface, and a subset are logged in the standard log file. |
|||
*Currently, MX does not calculate maximum and minima for any extra sensor measurements. |
|||
*For rainfall, MX does not use any rainfall readings supplied by the weather station, instead it works with a long term counter, and derives daily readings to output from subtracting one counter value (at end of required period) from another at start of day. For rain this month, Cumulus adds the current day's reading, and the readings logged in the [[Dayfile.txt|daily summary log]] for each preceding day in the month. For rainfall this year (also shown on the current conditions output, not any output for this year), the rainfall being output is the sum of the Year-to-date figure in the '''Settings''' menu, ''Station settings'' page, '''Annual rainfall''' section, today's rain, and the total rain stored in daily summary log for every previous day in the same calendar year. |
|||
*Depending on weather station (and settings selected), Cumulus may either use values from the weather station, or values it calculates itself, for items like Wind Chill and Forecasts. |
|||
*Weather stations do not supply Canadian Humidity Index (Humidex), USA Heat Index, Australian Apparent Temperature or the Feels Like temperature indices. For all these, Cumulus will calculate them by looking at various simultaneous readings (temperature, wind speed, and humidity for those two examples). |
|||
*The current conditions that Cumulus outputs are derived from every individual measurement, although if the weather station does not update between every output by Cumulus (e.g. Fine Offset that update only every 40 seconds), then Cumulus may repeat its previous readings for two outputs in succession (Cumulus will output at least every minute, sometimes every second if weather station updates that frequently). |
|||
**Cumulus provides recent history web tags to give you access to a selection of outputs for every minute in past 7 days. |
|||
==EXTREMES ISSUES== |
|||
Extremes appear on the records web page, these are highest/lowest in a month, in a year and all-time. |
|||
When MX is calculating those maximum and minimum derivatives, these are within an individual day, but they are compared against others in the same month (to update both the this month web tags and the monthly extremes web tags), against others in the same year (to update the this year web tags) and against all previous derivatives (to update the all-time web tags). I say update web tags, because that is how you view the extremes, but actually there is a log file for maximum and minimum in a day (today.ini), for this month (month.ini), for monthly-all-time (monthlyalltime.ini), for this year (year.ini), and for all time (alltime.ini). These log files are where MX stores the extremes and where you need to do corrections. For total rain this month, this year, maximum rain for this month in all years and highest rainfall (hourly, daily, monthly), the same log files are used. The highest daily wind run per month, per year, in the same month for all years and all-time is similarly stored in these log files and made available as web tags. |
|||
== Advice re problems == |
|||
#Dealing with input problems depends on your weather station type |
|||
#*If there is a problem with input from a Davis weather station, stop MX, then it may be if you reset the Davis console, and the logger you use with your David, it might stop the bad data from then on, this won't correct the past data, but hopefully will stop the problem |
|||
#*If there is a problem with an old style Fine Offset, often with rain or pressure reported, try stopping MX, then resetting the Fine Offset (some have a small switch below a small hole on the back of the console so use a strong long pointed instrument with a small diameter, maybe a small nail or a sewing tool to initiate a reset). Disconnecting the USB connection and removing a console battery may also reset a Fine Offset, or there may be some combination of touches or switch pushes that does a reset. |
|||
#* For other weather station types, look in the forum for advice or read the station instructions. |
|||
# If the output values are frequently incorrect, MX provides some calibration settings in the admin interface. These vary dependent on the type of weather reading you want to correct. In general they use a formula like <tt> Output = Multiplier_2 X Input<sup>2</sup> + Multiplier_1 X input + Offset</tt> so in the settings you can specify the multipliers and offset to use in that equation. |
|||
#If only a small numbers are incorrect, MX provides editors for the logs that store extremes (although for today only the rainfall can be edited). See the pages linked from [[:Category:Log Files|:Category:Log Files]] to discover more about each log and how to deal with rogue values. View and edit these on'''Edit''' tab of admin interface. |
|||
#You can correct any individual rogue value in a monthly log as MX (see viewer editor on '''Data logs''' tab of admin interface. |
|||
#As rainfall has a lot of different derivatives stored for it (hourly totals, daily totals, rain rate) it is quite complicated to correct, so there are a series of instruction in the [[FAQ#My_station_invented_some_rain_that_didn.27t_really_occur.2C_and_I_want_to_set_it_to_zero_.28or_some_other_figure.29|C1 FAQ]]. |
|||
= Problems with Cumulus MX use of MySQL database = |
|||
CumulusMX includes in the admin interface the ability to create tables, and to update the tables. It works with both a predetermined schema (tables named realtime, monthly, and dayfile by default), and with a schema chosen by the Cumulus user. The package also includes the executable '''ExportMySQL.exe''' that can add past data to two of the tables (monthly and dayfile). These features allow people with no knowledge of the Structured Query Language (SQL) to start using databases, and problems are highly likely, see if the advice below helps. |
|||
== Questions === |
|||
===What is my database name? == |
|||
Cumulus MX does not create a database. You must have an existing database for MX to create tables. If you don't know how to create a database, don't use this facility. If you have created a database, or your web provider has provide you with a database, then you should know the database names, what user names have what access, and what password is needed. |
|||
=== Can I host a database locally? === |
|||
Yes, the uploading that CumulusMX.exe and EXportMySQL.exe does will work on a local web server with a database as well as on a remote web server with a database. There are actually lots of packages that are free and will load Apache to run a web server, MySQL (or another database server) to allow you to use database tables, and a script language like PHP to allow you to generate web pages showing data from database queries. Then there are several packages that provides an easy way to manipulate a database (if you are using MySQL and PHP, then the most popular [but by no means the only] package has a name formed from those two with Admin added i.e.PHPMyAdmin; similar packages are available for other databases with the second part of name changing). |
|||
=== I'm updating my MX version and it says something about new columns === |
|||
Although SQL will work with XML organised language (where each data item is labelled, but also has a relationship with other items), it is most often used with "relational databases". A relational database is defined as one that stores all the information in one or more tables consisting of rows and columns. Furthermore, the order of the columns must be defined by a schema that also states what format is used in each table cell, but the rows can be stored in any order, and the user cannot control that order. |
|||
When a table is being updated the SQL normally names the columns that are to be updated. For that update to work the columns must be there, and the number of columns must match the number of values being supplied. If a particular version of MX adds new columns for the update, then the new columns must be defined and added to the table before the update can run. The release announcement may provide a SQL script to run to add the new columns, otherwise you need to use whatever administration package is made available by your web server provider. If you have PHP and MySQL, then PHPMyAdmin is one package often provided, but there are several alternatives, that may offer more. |
|||
=== I am using Cumulus 1 and I have existing tables, but I want to try MX === |
|||
#If the columns in your existing tables match those used by standard MX tables, then why not copy your existing tables (using a tool like PHPMyAdmin or whatever you have available) perhaps adding a "MX" suffix for the new name. Then in MX simply declare these table names via the Admin interface MySQL settings page. Set the host to the relevant name for your remote web server (or a local server), add username, password, alter settings to allow updates to leave your device, and alter settings on your web server to allow the MX device to update the database. '''Enable''' the relevant updates and MX should add more rows to the tables with your existing data. |
|||
#If there are extra columns in the MX schema compared to the schema you were using in Cumulus 1, follow instruction above as far as the necessary settings, but before you enable the updates either use any script provided in release post for when the extra columns were added, or use whatever administrative tool (it might be phpMyAdmin) is provide to add the extra columns. The enable the updates. |
|||
#If you want to use different columns to those MX uses (you don't have solar, or you want snow or extra sensor data to be in your database) then you need to use the Custom upload facilities that MX provides. To keep this answer short, I won't explain how here. |
|||
=== Using Custom SQL === |
|||
Obviously to use Custom SQL you need some knowledge of SQL and this answer cannot teach you SQL. But the general format of an update instruction for a whole row is given below, ensure the command you supply to MX has all the parts shown. The instructions in {} are optional, they just make the SQL read better, parts in lower case need to be replaced by whatever is appropriate for your table, values are web tags enclosed in single quotes e.g. <tt>'<#windrun>'</tt>, if there are any columns for which you don't have a value, use NULL without any quotes as this is SQL. |
|||
<pre> INSERT [INTO] database_table_name (primary_key_column_name, column_name_1, column_name_2, .... column_name_last) VALUES ('primary_key_value', 'web_tag_1', 'web_tag_2', ... 'web_tag_last');</pre> |
|||
= Problems with web pages or NOAA reports = |
= Problems with web pages or NOAA reports = |
||
Line 50: | Line 232: | ||
MX comes with a standard set of web pages. Although some look the same, these are different from the pages coming with C1. You may find that information labelled as for Cumulus 1 does apply to the MX web pages as for example they share the same cascading style sheet. |
MX comes with a standard set of web pages. Although some look the same, these are different from the pages coming with C1. You may find that information labelled as for Cumulus 1 does apply to the MX web pages as for example they share the same cascading style sheet. |
||
== Questions == |
== Questions and answers == |
||
=== How to set up a web site === |
=== How to set up a web site === |
||
Line 61: | Line 243: | ||
#Check your FTP server, does it allow delete, insert, and rename operations |
#Check your FTP server, does it allow delete, insert, and rename operations |
||
# Test using your own [[FTP_tools|FTP package]], have you got host, username, password, and directory specified correctly |
# Test using your own [[FTP_tools|FTP package]], have you got host, username, password, and directory specified correctly |
||
#Be aware that your web server hosting provider might change the ports you have to use, so check their documentation, or announcements and be prepared to change the port in MX settings. |
|||
#The settings required are on the Internet settings screen within the admin interface. |
#The settings required are on the Internet settings screen within the admin interface. |
||
#In the '''web/FTP site''' section, check the settings, and experiment with a different FTP protocol. |
#In the '''web/FTP site''' section, check the settings, and experiment with a different FTP protocol. For example MX can use standard FTP, or FTPS, and you can select whether to use TLS 1.2. Hopefuly whoever provides your web server has told you what FTP profile to use. |
||
#Be aware that there are differences between different MX versions on which of the following protocols are available: |
|||
#In the '''web/FTP settings''' section, check all the settings, some determine what is automatically uploaded, other settings should initially obey the tips shown on the settings screen. One item to try is changing the setting for ''Active FTP mode''. |
|||
#* Simple File Transfer Protocol (FTP) simple file transfer process; effectively the file is transferred exactly as it is without any attempt to apply any security to stop it being read if it is intercepted. |
|||
#*File Transfer Protocol Secure (FTPS), this adds Transport Level Security (the modern replacement for what was the Secure Sockets Layer (SSL), which is now prohibited by RFC7568); effectively this requires the network connection to be secured so sender and receiver only can see what has been transferred, it is hard to intercept anything sent over the network, let along read an individual file. |
|||
#*Secure File Transfer Program (SFTP), based on Secure Shell (SSH) protocol, an open standard for secure transfer over an unsecured network; effectively it can be intercepted, but only recipient can both read the file and remove the encrypting. |
|||
# As with any new functionality added at a particular version, it probably has not been tested for all the operating systems and other environmental conditions under which it can be used, and it is possible it might not work in the first version where it is introduced. |
|||
#In the '''web/FTP settings''' section, check all the settings, some determine what is automatically uploaded, other settings should initially obey the tips shown on the settings screen. One item to try is changing the setting for ''Active FTP mode''. |
|||
#Be careful you don't set the interval between successive FTP/FTPS/SFTP uploads too small. Each of these processes is a fairly slow way to transfer files, so if you are uploading large files it might take a little time, and you can't start a new upload until previous has finished. |
|||
#For most modern web pages, including those provided, use UTF-8 [[Webtags#Encoding|encoding description]]. If your web pages appear to be empty when they are uploaded, look near the front of such files where the [[Customised_templates#Coding_standard_and_character_encoding_for_Customised_Templates_using_the_same_names_as_Standard_Cumulus_Templates|encoding declarations]] will be found. Cumulus 1.9.4 build 1093 changed all the template pages from XHTML 1.0 Transitional using charset=iso-8859-1 encoding to HTML 5 using charset="UTF-8" encoding. It uses this without [http://en.wikipedia.org/wiki/Byte_order_mark Byte Order Mark] (BOM). |
#For most modern web pages, including those provided, use UTF-8 [[Webtags#Encoding|encoding description]]. If your web pages appear to be empty when they are uploaded, look near the front of such files where the [[Customised_templates#Coding_standard_and_character_encoding_for_Customised_Templates_using_the_same_names_as_Standard_Cumulus_Templates|encoding declarations]] will be found. Cumulus 1.9.4 build 1093 changed all the template pages from XHTML 1.0 Transitional using charset=iso-8859-1 encoding to HTML 5 using charset="UTF-8" encoding. It uses this without [http://en.wikipedia.org/wiki/Byte_order_mark Byte Order Mark] (BOM). |
||
#Look up any FTP error codes on the web. [http://kb.globalscape.com/KnowledgebaseArticle10142.aspx Try Here] |
#Look up any FTP error codes on the web. [http://kb.globalscape.com/KnowledgebaseArticle10142.aspx Try Here] |
||
Line 73: | Line 262: | ||
* On the ''Internet settings'' screen in '''Sites/Options''' the optional Directory parameter required is ''the path from your FTP root'', to where the web pages are to be stored. It is ''not'' the '''Universal Resource Locator (URL)''' that you would specify to view the web page in a browser. |
* On the ''Internet settings'' screen in '''Sites/Options''' the optional Directory parameter required is ''the path from your FTP root'', to where the web pages are to be stored. It is ''not'' the '''Universal Resource Locator (URL)''' that you would specify to view the web page in a browser. |
||
* Relative paths (without a leading slash) are always safer than absolute paths (with a leading slash) and you may find that just specifying (for example) "public_html" works OK, but this all depends on how your ftp server and your ftp account is set up. |
* Relative paths (without a leading slash) are always safer than absolute paths (with a leading slash) and you may find that just specifying (for example) "public_html" works OK, but this all depends on how your ftp server and your ftp account is set up. |
||
* The host name should be associated with a particular entry point into the web server, so any directory entered into the settings should be a path relative to that entry point. The entry point for FTP may be different to the entry point for displaying web pages. |
|||
=== |
=== Content missing from web server === |
||
This catch-all section covers everything from problems with not setting up web site to not uploading the right files at the interval you want them to be uploaded. |
|||
Please see [[FAQ#How_do_I_replace_the_.27bird.27_image_on_my_website.3F|C1 FAQ]]. |
|||
#All the files and folders ''within'' the '''CumulusMX/webfiles''' folder need to be transferred to your web server in a one-off operation. MX does not include a facility to do this. See the [[Website setup|setting up a web site]] page in this wiki. |
|||
#In the admin interface, settings tab, select the Internet settings screen, in the second section '''Web/FTP settings''', you can choose what MX does upload for you: |
|||
#*'''Auto update''' - this needs to be ticked for MX to upload anything for you at the standard processing and uploading interval |
|||
#*'''Include standard files''' - this is ticked for MX to process and upload the standard web templates to create web pages on your web server. The templates must be in the '''CumulusMX/web''' folder. |
|||
#**Cumulus 1 in earlier versions created an empty web page if any of the standard web templates did not exist |
|||
#**Cumulus 1 in later versions does not attempt to upload a web page if the relevant web template does not exist |
|||
#**Cumulus MX will issue an error message if, when it tries to process all of the web templates, any are missing |
|||
#*'''Include graph data files''' - this is ticked if you want MX to generate JSON files (these will be created in the '''CumulusMX/web''' folder, so that folder must exist) and upload them to your web server (at the Realtime interval if that is enabled lower down this screen) |
|||
#*'''Include Moon image file''' - there is a separate setting that controls whether MX creates updated moon images each hour, but this setting controls whether that image is uploaded to your web server each time MX updates the image. |
|||
#* Somewhat lower down the screen is the '''Upload interval''' text box where you specify the interval between standard uploads, it is an error to make it blank or set it to zero! The default is every 15 minutes. |
|||
#*'''Enable Realtime''' option, if you don't tick that, either the standard uploading interval will be used for files that can be processed at this interval, or the files will not be processed. |
|||
#*'''Enable realtime FTP''' - this option is ticked to get MX to use FTP to transfer any files you select to be updated at this interval |
|||
#*'''Realtimegauges.txt''' - this option is ticked for MX to create and upload the file that the Gauges web page provided with MX needs for it to actually display your data. |
|||
#*'''Realtime interval''' - in this text box you can edit the time between real time uploads. Don't set it to too small a number, think how often your station issues new readings, how quickly the device you are running MX on can do a FTP upload, and how often you need to see updates on your web site. |
|||
=== The bird image on standard web pages === |
|||
#If you don't see this with standard web page set up then you have missed out the step of uploading all files and folders within the '''webfiles''' folder in the distribution just once to your web site. The contents of this folder are not used locally, you don't need to keep the folder after it has been uploaded to your web server (even if that is a local web server). |
|||
#If you don't want to see the bird image, please see [[FAQ#How_do_I_replace_the_.27bird.27_image_on_my_website.3F|C1 FAQ]]. |
|||
=== I want to remove the solar data from the supplied web templates === |
=== I want to remove the solar data from the supplied web templates === |
||
Please see [[FAQ#I_don.27t_have_a_solar_sensor._How_do_I_remove_the_solar_data_from_the_supplied_web_templates.3F|C1 FAQ]] |
Please see [[FAQ#I_don.27t_have_a_solar_sensor._How_do_I_remove_the_solar_data_from_the_supplied_web_templates.3F|C1 FAQ]] |
||
===Changing content within standard web templates === |
|||
See previous answer, you can make any change you like on the template pages, when MX processes the template any change you have made will appear in web page that MX next uploads to your web server. |
|||
For more guidance, please see [[Customised templates]] page that takes you through all possibilities. |
|||
=== How do I remove Inside Temperature graph or add solar/UV/sunshine? === |
=== How do I remove Inside Temperature graph or add solar/UV/sunshine? === |
||
Line 91: | Line 307: | ||
The styling for these headings, including use of lower case, is defined in the '''C'''ascading '''S'''tyling '''S'''heet. You need some understanding of [[Stylesheets|CSS]] to edit these. |
The styling for these headings, including use of lower case, is defined in the '''C'''ascading '''S'''tyling '''S'''heet. You need some understanding of [[Stylesheets|CSS]] to edit these. |
||
=== I have a customised web template with yesterday web tags === |
|||
Set this up via the '''Extra Web Files''' settings in the Admin interface. Use only static and yesterday web tags in your template, type the local path/file name in local column, tick the process column, tick the UTF-8 column if you declare that encoding in your template (see [[Customised templates]] for examples), tick the EOD column and put the path to where you want the web page to be created in remote column. |
|||
=== The sunrise and sunset times are wrong === |
=== The sunrise and sunset times are wrong === |
||
Line 118: | Line 338: | ||
#Changing the [[Webtags#Encoding|encoding]] on NOAA reports can cause problems, especially if you try to display them from a web page that does not share the same encoding. |
#Changing the [[Webtags#Encoding|encoding]] on NOAA reports can cause problems, especially if you try to display them from a web page that does not share the same encoding. |
||
#Making an error in the report naming parameters can cause problems, see [[Webtags#NOAA_style_Report_Naming|NOAA_style_Report_Naming]] section. |
#Making an error in the report naming parameters can cause problems, see [[Webtags#NOAA_style_Report_Naming|NOAA_style_Report_Naming]] section. |
||
#MX creates reports in a folder called "Reports" with an initial capital letter. That folder is part of the distribution in every MX release, and you must have that folder, not is equivalent in all lower case. |
|||
#There is a setting to determine whether NOAA reports are generated (at end of day), and another setting to control uploading to your web server (you need to specify the directory to upload to). Check these settings are all correct. |
|||
=== I have problems with my own web templates that Cumulus needs to process === |
=== I have problems with my own web templates that Cumulus needs to process === |
||
Line 132: | Line 354: | ||
# You will find advice on how to choose what language to use for scripts on [[PHP|PHP and Javascript]] page, along with references to some easy PHP scripts by Cumulus users. |
# You will find advice on how to choose what language to use for scripts on [[PHP|PHP and Javascript]] page, along with references to some easy PHP scripts by Cumulus users. |
||
# That advice continues with how to use a suite of PHP scripts to populate PHP variables with values taken from web tags at [[Php webtags|PHP Hypertext Preprocessor scripts reading web tags]]. |
# That advice continues with how to use a suite of PHP scripts to populate PHP variables with values taken from web tags at [[Php webtags|PHP Hypertext Preprocessor scripts reading web tags]]. |
||
=== What file names are used when MX processes files === |
|||
[[File:Cumulus_FTP_Process.png|350px|right|FTP Process]]Cumulus use two slightly different processes depending on if the file is part of the supplied 'standard' web site, or an extra file you have specified on the ''Internet Settings|Files'' configuration dialog. For each of these processes there is an further variation depending on whether you have enabled the ''Use FTP rename'' option. This graphic shows the various combinations of file-names used for each step of the process. For the standard files, you can substitute any of the file names for indexT.htm, e.g. trendsT.htm |
|||
For more details of Cumulus ''Process'' see [[Customised_templates#What_is_meant_by_.27Cumulus_processes_templates.27| here]]. |
|||
=== I use web page packages provided by third parties === |
=== I use web page packages provided by third parties === |
||
By third parties I mean items like those listed on [[Category:User Contributions|User Contributions]] page. |
By third parties I mean items like those listed on [[:Category:User Contributions|User Contributions]] page. |
||
For most problems you probably need to contact the author via the support forum, but here are some possible answers |
For most problems you probably need to contact the author via the support forum, but here are some possible answers |
||
# You did not follow the instructions supplied for using that package, and you have used wrong file names or not put all the files where they should be or got confused between one-off transfers to your web site and files that Cumulus needs to upload at various intervals. |
# You did not follow the instructions supplied for using that package, and you have used wrong file names or not put all the files where they should be or got confused between one-off transfers to your web site and files that Cumulus needs to upload at various intervals. |
||
# File names in that package conflict with file names you already have on your server (this might stop any PHP require/includes linking in correct file) |
# File names in that package conflict with file names you already have on your server (this might stop any PHP require/includes linking in correct file) |
||
# Maybe you are mixing some items using "http:" protocol and some using "https:", you need to |
# Maybe you are mixing some items using "http:" protocol and some using "https:", you need to choose one or the other, not a mixture. |
||
# Perhaps the package depends on what version of JavaScript, PHP, MySQL, you have installed, or on some setting for those languages. Depending on your web server, and the sort of access provide by any web provider you use , you might have a "cPanel" where these can be changed, or maybe you have to ask the provider to do the necessary changes. |
# Perhaps the package depends on what version of JavaScript, PHP, MySQL, you have installed, or on some setting for those languages. Depending on your web server, and the sort of access provide by any web provider you use , you might have a "cPanel" where these can be changed, or maybe you have to ask the provider to do the necessary changes. |
||
Latest revision as of 18:17, 26 January 2021
Introduction
If you are seeking information on why you might want to use Cumulus, then please read the About Cumulus article first. That is a general introduction to what Cumulus does and what might be its advantages over alternative software.
It is anticipated that the main documentation for Cumulus MX will be placed in this article and you should read that to get basic information about MX.
This article is to answer those awkward questions that will arise when you try to go beyond the "normal" installation or you need some trouble shooting style help. The questions have been placed into groups, but this is my first attempt at designing this article, so please feel free to make improvements that suit the way you search.
Request for assistance
If you, either need help on topics not covered in this article, or you see any messages that are not explained below, please update this article (you may need to ask for a login) or else post on the support forum in Cumulus Wiki Suggestions for updates and additions sub-forum. Thank you.
Background
Mark Crossley created this page on 26 October 2018 to help people with questions about MX where it works differently to Cumulus 1 and the Cumulus 1 FAQ did not help.
The original content covered device names (COMPORT in MX has different format to Port in C1), Windows 10 issues, and MONO issues. But my impression is nobody found this, and therefore nobody benefited from that text.
I started pasting in useful material, mostly from Steve Loft's posts in the forum about the beta version of MX, but as at that time I was not using MX I hoped somebody else would make sense of it and create a better layout. Of course nobody did.
When I did start using MX, I decided to edit other articles first, see Cumulus MX, Cumulus MX formal release versions, Webtags, Cumulus.ini, Dayfile.txt, Category:Log Files, Php webtags, and various others.
So that left this page, but I don't have enough time to complete updating this article. If you read this, and believe you can help with completing this update, please do.
Installation and Upgrading problems
Basic advice
- Whether you are installing MX for the first time, or updating to a new version, you download the zipped software from Software page.
- Unzip it either directly to the place where you want it, or to a holding area where you keep downloaded software, and then use copy or another file transfer process to get it to final destination
- You will find more advice in the Cumulus_MX documentation page.
- If there is a problem, the most likely problem is that you are trying to install onto say a SD card and that has failed, and needs to be replaced. You might want to consider investing in a SSD drive, they tend to be very reliable.
I use Cumulus 1 and want to move to Cumulus MX
Please see Moving from Cumulus 1 to MX article.
Variation between releases
- Cumulus MX is currently being developed
- Some functionality changes between version
- Some bugs probably remain from the older beta versions of MX initiated by Steve Loft.
- Each new piece of functionality has potential to add further bugs as it is impossible for the developer to test that feature with all possible weather station types, and with all setting permutations
- Most releases are tested by a few beta testers prior to them being released generally, but again these people don't test all permutations.
- Consequently, if you always upgrade to the latest version, you might encounter new functionality that possibly might not work with your particular set-up.
- Equally, if you don't always update to the latest version, you might miss out on a solution to a problem in the version you are using
- The advice is therefore to retain the download of the version you are currently using when you upgrade to a new version, so regression is easy if you do encounter a problem with new functionality.
- Always read the release notice when a new release is available, that might determine the speed at which you should upgrade.
If you ever do need to download the distribution for a release prior to the current one, please go to this web page. Don't forget to read the release notices on the forum, because there might be differences between releases that require action by you (whether increasing version number or decreasing version number).
Troubleshooting - messages in MXdiags or in console/terminal output
Mark Crossley lists the issues outstanding in beta version of MX in MX Issues article. Although that has not been updated since 2018, it might give you a hint of bugs that might still exist.
MESSAGE FROM SFWS
I do not have a separate environment to test MX in, and I don't want to mess up my set of weather measurements by any action causing MX to give an error message just to be able to get this article right.
Also I don't want to search through all the source code to look up every possible message that MX can produce, I know different versions of MX can differ in what messages they output to the diags, so it is not simple to know what to include here. It may be I have matched the message MX gives to the wrong solution because I do not have a complete knowledge of how MX works. Please do correct any mistakes I have made and improve this article, I cannot do it all alone.
So forgive me if actual MX messages are different to those I quote here, I just hope that you find what I have typed here helpful.
If you see another message, please add the message and solution here, so it becomes a better place to help people solve any problems they encounter.
Questions and answers
Not watching the terminal or console window
Although the MX engine writes messages to the terminal or console (the terminology varies depending on the device) many people don't have a monitor connected to the device running MX (and if they do, they don't watch this output). Therefore it is easy to not to see messages, and not to know when MX is running, miss out when it is reporting errors, or perhaps even to fail to spot when it crashes.
One problem people have is stopping MX while it is busy, perhaps still doing catch-up from your station's logger after restarting, perhaps doing an end of day or end of month rollover, maybe in the middle of some other task. It is not so easy to see whether MX is running, nor when it is busy. MX does so many tasks now, that picking a moment when it is not doing something is harder.
Although MX is designed assuming it is left running all the time, it is permissible to stop, and restart it, just do that as few times as possible (once a day if you must). MX will run on a variety of devices, so can run on devices that are cheap to leave on all the time.
Station type not known
When you first run MX, it needs to be told what station type to look for, and how to connect to that station. Start up the Administrative Interface and select the settings tab, it should then display station settings for you to make the necessary choices.
Unable to connect to station
This means that you have not set all the settings mentioned in previous answer.
- Maybe you have selected a station type where you need to select a connection port
- Maybe you have migrated from Cumulus 1 to MX. These two flavours use a different setting to select the port, so MX ignores the C1 setting (Port=number in [Station] section of Cumulus.ini configuration file) and you need to enter the one MX uses (Comport=text in Cumulus.ini).
Weather Station Problems
If one of the sensors on your weather station fails, MX will reuse the same value for 6 reads, after that it will conclude the station is fault, and with default settings, MX will stop working.
There are many places where you can look on the web for advice on setting up a weather station. In the UK, the best site to read first is the resources linked from how-can-you-make-high-quality-weather-observations page.
The Moon rise or set time is showing dashes.
The moon doesn’t rise and set every day, the dashes appear when that event doesn’t occur on that particular day. The period between a moon rise and set is approximately 12 hours and 25 minutes, on average. The moon rises (or sets) approximately 50 minutes later, on average, each day. It therefore follows that sometimes the moon will not rise or set on a particular day - i.e. it had risen the previous day, or will set on the following day.
The Moon image is just black
It's probably a New Moon.
I have a message about web tags
There is an article about Web tags, I have tried to keep that up to date. That may help, the most common errors seem to be:
- Using a web tag not available at the version you are using (the message in the MXDiags should indicate which one).
- Using the wrong output formatters, as the article referenced above explains, the codes to use for MX are far more complicated.
- Using an input modifier that MX cannot match to data available (the message will talk about arrays).
I have a message about pressing Enter because a problem has occurred
This could cover a lot of problems. Make sure you don't have 2 copies of MX running and are not running anything else that accesses the same weather station or the same files as MX, because this message may relate to such a conflict.
Maybe there is some fault with your weather station to MX communication.
Other problems that crash MX when you have just updated either MX or Mono, may relate to code added in update. For example 3.6.2 crashed in some contexts because of a new "FluentFTP.dll file", 3.6.3 reverted to the one used for 3.6.0. Other people have seen problems because of changes in how MONO worked on an update, although that was more of a problem with MX beta than later MX.
Maybe MX is trying to upload to your web server and that web server is not currently available. Or MX is trying to upload to a database server, and that is not currently available.
I have a message about missing web templates
If you have selected the Include standard files option, MX expects to find all the standard web templates in the folder CumulusMX/web, and will give an error message if it cannot find that folder or cannot find all the files in that folder that are provided in each MX distribution.
Accessing the Admin Interface
The admin interface is where you set all settings for MX. It also provides screens for editing log files and the extremes they contain. Finally it provides a number of pages where you can view the weather data Cumulus MX is processing.
Questions
Most problems will be in gaining access.
- Read the documentation first to see how to access this interface.
- Because the MX engine generates a web server on which to run the interface, there might be a permissions problem
- You might need to issue a one-off command to let the MX engine generate the web server, this often involves giving it root or administrator privileges although it does not need to run at the root or run in an administrator user. The command needed depends on your operating system, see the documentation, to see if it is covered there.
- You may need to edit settings in your virus checker, firewall, or similar software to open the 8998 port (or if you need to use a different port, start MX with the relevant port parameter).
Problems with MX not accepting data when move from C1
Questions
Format of data files
Please see Log Files index page for more information. In general, you should have all the files (except log.xml from your C1 installation in your MX installation, and MX will be able to read them, although it may update content and thus stop C1 from recognising them. Ensure you have same culture as before (so don't change how dates/times and decimals are stored).
Ignoring earlier data
There are two contexts where new Cumulus MX users have reported problems stopping them seeing some earlier data they saw when they were using Cumulus 1 (C1).
- MX needs to read from Cumulus.ini the earliest date to process as it ignores any content for earlier dates. (Cumulus 1 just read all the files it could find in the data folder.
- When you first start running MX, it does not seem to be able to populate charts and recent history web tags for earlier times.
Dealing with rogue values
INPUT ISSUES
Input issues are when Cumulus reads the wrong information from the weather station.
- Sometimes a weather station will supply incorrect data bits that mean wrong numbers are input to Cumulus MX or that MX looking at byte level does not recognise what it is being sent
- What Cumulus outputs, for up to 3 different wind speeds it can calculate, depends on the weather station and which outputs from the weather station are processed by Cumulus.
- For Fine Offset (not Ecowitt), MX has various options on Settings menu, Calibration settings page, Spike removal section to reject any readings that differ from previous value by more than figure selected. As mentioned, this is not available for most weather stations, only those that can optionally output to Easyweather software.
- For all weather stations, MX has various options on Settings menu, Calibration settings page, offsets and multiplier sections to calibrate what is read from the weather station by applying a multiplier (in some cases both first and second order multipliers), and/or an offset.
- MX can also reuse the last good reading (for up to 6 times) if there is a problem with individual readings. After that it gives up and labels that sensor as not working (see next point).
- MX can stop running if it is unable to obtain readings for key measurements, i.e. temperature, pressure, wind speed, humidity, although there are some settings that can over-ride this action.
DERIVATIVES ISSUES
Derivatives are any values that Cumulus reports which it has not read directly from the weather station. In a sense, every out put by Cumulus is a derivative, because the units that the weather station supplies readings in, have to be converted to the units that the user has requested.
- Cumulus MX ignores any maximum and minimum values supplied by a weather station; it calculates these highest and lowest derived values each metrological day for itself, from every measurement it processes, not the small sample of it actually writes to the standard log.
- As each new standard sensor reading is read in it is first converted to units selected by user, then the reading is added to the recent history array, then it is compared against the previous highest/lowest for that meteorological day, updating those as needed, and finally the reading is placed into web tags (for web pages), application programming interface that can feed it to admin interface, and a subset are logged in the standard log file.
- Currently, MX does not calculate maximum and minima for any extra sensor measurements.
- For rainfall, MX does not use any rainfall readings supplied by the weather station, instead it works with a long term counter, and derives daily readings to output from subtracting one counter value (at end of required period) from another at start of day. For rain this month, Cumulus adds the current day's reading, and the readings logged in the daily summary log for each preceding day in the month. For rainfall this year (also shown on the current conditions output, not any output for this year), the rainfall being output is the sum of the Year-to-date figure in the Settings menu, Station settings page, Annual rainfall section, today's rain, and the total rain stored in daily summary log for every previous day in the same calendar year.
- Depending on weather station (and settings selected), Cumulus may either use values from the weather station, or values it calculates itself, for items like Wind Chill and Forecasts.
- Weather stations do not supply Canadian Humidity Index (Humidex), USA Heat Index, Australian Apparent Temperature or the Feels Like temperature indices. For all these, Cumulus will calculate them by looking at various simultaneous readings (temperature, wind speed, and humidity for those two examples).
- The current conditions that Cumulus outputs are derived from every individual measurement, although if the weather station does not update between every output by Cumulus (e.g. Fine Offset that update only every 40 seconds), then Cumulus may repeat its previous readings for two outputs in succession (Cumulus will output at least every minute, sometimes every second if weather station updates that frequently).
- Cumulus provides recent history web tags to give you access to a selection of outputs for every minute in past 7 days.
EXTREMES ISSUES
Extremes appear on the records web page, these are highest/lowest in a month, in a year and all-time.
When MX is calculating those maximum and minimum derivatives, these are within an individual day, but they are compared against others in the same month (to update both the this month web tags and the monthly extremes web tags), against others in the same year (to update the this year web tags) and against all previous derivatives (to update the all-time web tags). I say update web tags, because that is how you view the extremes, but actually there is a log file for maximum and minimum in a day (today.ini), for this month (month.ini), for monthly-all-time (monthlyalltime.ini), for this year (year.ini), and for all time (alltime.ini). These log files are where MX stores the extremes and where you need to do corrections. For total rain this month, this year, maximum rain for this month in all years and highest rainfall (hourly, daily, monthly), the same log files are used. The highest daily wind run per month, per year, in the same month for all years and all-time is similarly stored in these log files and made available as web tags.
Advice re problems
- Dealing with input problems depends on your weather station type
- If there is a problem with input from a Davis weather station, stop MX, then it may be if you reset the Davis console, and the logger you use with your David, it might stop the bad data from then on, this won't correct the past data, but hopefully will stop the problem
- If there is a problem with an old style Fine Offset, often with rain or pressure reported, try stopping MX, then resetting the Fine Offset (some have a small switch below a small hole on the back of the console so use a strong long pointed instrument with a small diameter, maybe a small nail or a sewing tool to initiate a reset). Disconnecting the USB connection and removing a console battery may also reset a Fine Offset, or there may be some combination of touches or switch pushes that does a reset.
- For other weather station types, look in the forum for advice or read the station instructions.
- If the output values are frequently incorrect, MX provides some calibration settings in the admin interface. These vary dependent on the type of weather reading you want to correct. In general they use a formula like Output = Multiplier_2 X Input2 + Multiplier_1 X input + Offset so in the settings you can specify the multipliers and offset to use in that equation.
- If only a small numbers are incorrect, MX provides editors for the logs that store extremes (although for today only the rainfall can be edited). See the pages linked from :Category:Log Files to discover more about each log and how to deal with rogue values. View and edit these onEdit tab of admin interface.
- You can correct any individual rogue value in a monthly log as MX (see viewer editor on Data logs tab of admin interface.
- As rainfall has a lot of different derivatives stored for it (hourly totals, daily totals, rain rate) it is quite complicated to correct, so there are a series of instruction in the C1 FAQ.
Problems with Cumulus MX use of MySQL database
CumulusMX includes in the admin interface the ability to create tables, and to update the tables. It works with both a predetermined schema (tables named realtime, monthly, and dayfile by default), and with a schema chosen by the Cumulus user. The package also includes the executable ExportMySQL.exe that can add past data to two of the tables (monthly and dayfile). These features allow people with no knowledge of the Structured Query Language (SQL) to start using databases, and problems are highly likely, see if the advice below helps.
Questions =
=What is my database name?
Cumulus MX does not create a database. You must have an existing database for MX to create tables. If you don't know how to create a database, don't use this facility. If you have created a database, or your web provider has provide you with a database, then you should know the database names, what user names have what access, and what password is needed.
Can I host a database locally?
Yes, the uploading that CumulusMX.exe and EXportMySQL.exe does will work on a local web server with a database as well as on a remote web server with a database. There are actually lots of packages that are free and will load Apache to run a web server, MySQL (or another database server) to allow you to use database tables, and a script language like PHP to allow you to generate web pages showing data from database queries. Then there are several packages that provides an easy way to manipulate a database (if you are using MySQL and PHP, then the most popular [but by no means the only] package has a name formed from those two with Admin added i.e.PHPMyAdmin; similar packages are available for other databases with the second part of name changing).
I'm updating my MX version and it says something about new columns
Although SQL will work with XML organised language (where each data item is labelled, but also has a relationship with other items), it is most often used with "relational databases". A relational database is defined as one that stores all the information in one or more tables consisting of rows and columns. Furthermore, the order of the columns must be defined by a schema that also states what format is used in each table cell, but the rows can be stored in any order, and the user cannot control that order.
When a table is being updated the SQL normally names the columns that are to be updated. For that update to work the columns must be there, and the number of columns must match the number of values being supplied. If a particular version of MX adds new columns for the update, then the new columns must be defined and added to the table before the update can run. The release announcement may provide a SQL script to run to add the new columns, otherwise you need to use whatever administration package is made available by your web server provider. If you have PHP and MySQL, then PHPMyAdmin is one package often provided, but there are several alternatives, that may offer more.
I am using Cumulus 1 and I have existing tables, but I want to try MX
- If the columns in your existing tables match those used by standard MX tables, then why not copy your existing tables (using a tool like PHPMyAdmin or whatever you have available) perhaps adding a "MX" suffix for the new name. Then in MX simply declare these table names via the Admin interface MySQL settings page. Set the host to the relevant name for your remote web server (or a local server), add username, password, alter settings to allow updates to leave your device, and alter settings on your web server to allow the MX device to update the database. Enable the relevant updates and MX should add more rows to the tables with your existing data.
- If there are extra columns in the MX schema compared to the schema you were using in Cumulus 1, follow instruction above as far as the necessary settings, but before you enable the updates either use any script provided in release post for when the extra columns were added, or use whatever administrative tool (it might be phpMyAdmin) is provide to add the extra columns. The enable the updates.
- If you want to use different columns to those MX uses (you don't have solar, or you want snow or extra sensor data to be in your database) then you need to use the Custom upload facilities that MX provides. To keep this answer short, I won't explain how here.
Using Custom SQL
Obviously to use Custom SQL you need some knowledge of SQL and this answer cannot teach you SQL. But the general format of an update instruction for a whole row is given below, ensure the command you supply to MX has all the parts shown. The instructions in {} are optional, they just make the SQL read better, parts in lower case need to be replaced by whatever is appropriate for your table, values are web tags enclosed in single quotes e.g. '<#windrun>', if there are any columns for which you don't have a value, use NULL without any quotes as this is SQL.
INSERT [INTO] database_table_name (primary_key_column_name, column_name_1, column_name_2, .... column_name_last) VALUES ('primary_key_value', 'web_tag_1', 'web_tag_2', ... 'web_tag_last');
Problems with web pages or NOAA reports
MX comes with a standard set of web pages. Although some look the same, these are different from the pages coming with C1. You may find that information labelled as for Cumulus 1 does apply to the MX web pages as for example they share the same cascading style sheet.
Questions and answers
How to set up a web site
Please see Web site setup article.
My FTP is not working
- Check your firewall package, does it allow delete, insert, and rename operations
- Check your FTP server, does it allow delete, insert, and rename operations
- Test using your own FTP package, have you got host, username, password, and directory specified correctly
- Be aware that your web server hosting provider might change the ports you have to use, so check their documentation, or announcements and be prepared to change the port in MX settings.
- The settings required are on the Internet settings screen within the admin interface.
- In the web/FTP site section, check the settings, and experiment with a different FTP protocol. For example MX can use standard FTP, or FTPS, and you can select whether to use TLS 1.2. Hopefuly whoever provides your web server has told you what FTP profile to use.
- Be aware that there are differences between different MX versions on which of the following protocols are available:
- Simple File Transfer Protocol (FTP) simple file transfer process; effectively the file is transferred exactly as it is without any attempt to apply any security to stop it being read if it is intercepted.
- File Transfer Protocol Secure (FTPS), this adds Transport Level Security (the modern replacement for what was the Secure Sockets Layer (SSL), which is now prohibited by RFC7568); effectively this requires the network connection to be secured so sender and receiver only can see what has been transferred, it is hard to intercept anything sent over the network, let along read an individual file.
- Secure File Transfer Program (SFTP), based on Secure Shell (SSH) protocol, an open standard for secure transfer over an unsecured network; effectively it can be intercepted, but only recipient can both read the file and remove the encrypting.
- As with any new functionality added at a particular version, it probably has not been tested for all the operating systems and other environmental conditions under which it can be used, and it is possible it might not work in the first version where it is introduced.
- In the web/FTP settings section, check all the settings, some determine what is automatically uploaded, other settings should initially obey the tips shown on the settings screen. One item to try is changing the setting for Active FTP mode.
- Be careful you don't set the interval between successive FTP/FTPS/SFTP uploads too small. Each of these processes is a fairly slow way to transfer files, so if you are uploading large files it might take a little time, and you can't start a new upload until previous has finished.
- For most modern web pages, including those provided, use UTF-8 encoding description. If your web pages appear to be empty when they are uploaded, look near the front of such files where the encoding declarations will be found. Cumulus 1.9.4 build 1093 changed all the template pages from XHTML 1.0 Transitional using charset=iso-8859-1 encoding to HTML 5 using charset="UTF-8" encoding. It uses this without Byte Order Mark (BOM).
- Look up any FTP error codes on the web. Try Here
The following pointers should help:
- All web site file references are case sensitive. So if your FTP server directory is public_html, then 'Public_HTML' will not work!
- On the Internet settings screen in Sites/Options the optional Directory parameter required is the path from your FTP root, to where the web pages are to be stored. It is not the Universal Resource Locator (URL) that you would specify to view the web page in a browser.
- Relative paths (without a leading slash) are always safer than absolute paths (with a leading slash) and you may find that just specifying (for example) "public_html" works OK, but this all depends on how your ftp server and your ftp account is set up.
- The host name should be associated with a particular entry point into the web server, so any directory entered into the settings should be a path relative to that entry point. The entry point for FTP may be different to the entry point for displaying web pages.
Content missing from web server
This catch-all section covers everything from problems with not setting up web site to not uploading the right files at the interval you want them to be uploaded.
- All the files and folders within the CumulusMX/webfiles folder need to be transferred to your web server in a one-off operation. MX does not include a facility to do this. See the setting up a web site page in this wiki.
- In the admin interface, settings tab, select the Internet settings screen, in the second section Web/FTP settings, you can choose what MX does upload for you:
- Auto update - this needs to be ticked for MX to upload anything for you at the standard processing and uploading interval
- Include standard files - this is ticked for MX to process and upload the standard web templates to create web pages on your web server. The templates must be in the CumulusMX/web folder.
- Cumulus 1 in earlier versions created an empty web page if any of the standard web templates did not exist
- Cumulus 1 in later versions does not attempt to upload a web page if the relevant web template does not exist
- Cumulus MX will issue an error message if, when it tries to process all of the web templates, any are missing
- Include graph data files - this is ticked if you want MX to generate JSON files (these will be created in the CumulusMX/web folder, so that folder must exist) and upload them to your web server (at the Realtime interval if that is enabled lower down this screen)
- Include Moon image file - there is a separate setting that controls whether MX creates updated moon images each hour, but this setting controls whether that image is uploaded to your web server each time MX updates the image.
- Somewhat lower down the screen is the Upload interval text box where you specify the interval between standard uploads, it is an error to make it blank or set it to zero! The default is every 15 minutes.
- Enable Realtime option, if you don't tick that, either the standard uploading interval will be used for files that can be processed at this interval, or the files will not be processed.
- Enable realtime FTP - this option is ticked to get MX to use FTP to transfer any files you select to be updated at this interval
- Realtimegauges.txt - this option is ticked for MX to create and upload the file that the Gauges web page provided with MX needs for it to actually display your data.
- Realtime interval - in this text box you can edit the time between real time uploads. Don't set it to too small a number, think how often your station issues new readings, how quickly the device you are running MX on can do a FTP upload, and how often you need to see updates on your web site.
The bird image on standard web pages
- If you don't see this with standard web page set up then you have missed out the step of uploading all files and folders within the webfiles folder in the distribution just once to your web site. The contents of this folder are not used locally, you don't need to keep the folder after it has been uploaded to your web server (even if that is a local web server).
- If you don't want to see the bird image, please see C1 FAQ.
I want to remove the solar data from the supplied web templates
Please see C1 FAQ
Changing content within standard web templates
See previous answer, you can make any change you like on the template pages, when MX processes the template any change you have made will appear in web page that MX next uploads to your web server.
For more guidance, please see Customised templates page that takes you through all possibilities.
How do I remove Inside Temperature graph or add solar/UV/sunshine?
Beteljuice has written a routine to change Highstock graphs in MX that is available here.
I want a different heading on standard web pages
These are built up from a combination of text in the template file and what you have entered on the station settings in text boxes just above the Options section.
The styling for these headings, including use of lower case, is defined in the Cascading Styling Sheet. You need some understanding of CSS to edit these.
I have a customised web template with yesterday web tags
Set this up via the Extra Web Files settings in the Admin interface. Use only static and yesterday web tags in your template, type the local path/file name in local column, tick the process column, tick the UTF-8 column if you declare that encoding in your template (see Customised templates for examples), tick the EOD column and put the path to where you want the web page to be created in remote column.
The sunrise and sunset times are wrong
- You must enter DEGREES, MINUTES, and SECONDS, in the text boxes within the location section and select hemisphere, in the Station settings in the admin interface. See C1 FAQ for more advice.
- Another possible cause might be your device system date and time is wrong.
The moon rise or moon set shows dashes
This is common, the lunar day does not match the earth day, so on some days the moon will either only rise or only set, while other days will see a moon rise and a moon set.
I want a moon image, but I don't see one
- Are you using an old version that does not include a moon image?
- Do you have the moon base image in your local files?
- Have you asked MX to generate by system drawing a masked image to show the current phase?
- Have you asked MX to upload this image?
- Is your FTP working, because that is only way the moon image can be uploaded to web server.
I modified the standard web pages, and since I updated to a new MX version I have lost my edits
In MX you can choose to either overwrite all files when there is a new release or to overwrite only files that you have not changed. In general, it is best not to modify the files provided with MX, instead copy them to a new folder and edit them there. For web templates, if they are in a different folder, you will need to specify them in the local column of Extra web files and turn off the option to upload standard web pages. For more information on customising web pages seeCustomised templates article.
I have problems with NOAA reports
- Editing Cumulus.ini without stopping MX can cause problems
- Changing the encoding on NOAA reports can cause problems, especially if you try to display them from a web page that does not share the same encoding.
- Making an error in the report naming parameters can cause problems, see NOAA_style_Report_Naming section.
- MX creates reports in a folder called "Reports" with an initial capital letter. That folder is part of the distribution in every MX release, and you must have that folder, not is equivalent in all lower case.
- There is a setting to determine whether NOAA reports are generated (at end of day), and another setting to control uploading to your web server (you need to specify the directory to upload to). Check these settings are all correct.
I have problems with my own web templates that Cumulus needs to process
- If your problems relate to web tags see earlier answer about error messages in MXDiags
- If you have problems with customising web pages, see reference in earlier answer
- If you have problems with selections in "Extra web files", see Extra_Web_Files.
- If you have problems with layout please see Web page layout article.
I am trying to use PHP Hypertext Pre-processor
PHP is a language for script that can do various tasks, but frequently it is used to prepare Hyper Text Manipulation Language file (HTML) to generate a web page, perhaps allowing the content to be varied by including some content generated from PHP variables or calculations.
- You will find advice on how to choose what language to use for scripts on PHP and Javascript page, along with references to some easy PHP scripts by Cumulus users.
- That advice continues with how to use a suite of PHP scripts to populate PHP variables with values taken from web tags at PHP Hypertext Preprocessor scripts reading web tags.
What file names are used when MX processes files
Cumulus use two slightly different processes depending on if the file is part of the supplied 'standard' web site, or an extra file you have specified on the Internet Settings|Files configuration dialog. For each of these processes there is an further variation depending on whether you have enabled the Use FTP rename option. This graphic shows the various combinations of file-names used for each step of the process. For the standard files, you can substitute any of the file names for indexT.htm, e.g. trendsT.htm
For more details of Cumulus Process see here.
I use web page packages provided by third parties
By third parties I mean items like those listed on User Contributions page.
For most problems you probably need to contact the author via the support forum, but here are some possible answers
- You did not follow the instructions supplied for using that package, and you have used wrong file names or not put all the files where they should be or got confused between one-off transfers to your web site and files that Cumulus needs to upload at various intervals.
- File names in that package conflict with file names you already have on your server (this might stop any PHP require/includes linking in correct file)
- Maybe you are mixing some items using "http:" protocol and some using "https:", you need to choose one or the other, not a mixture.
- Perhaps the package depends on what version of JavaScript, PHP, MySQL, you have installed, or on some setting for those languages. Depending on your web server, and the sort of access provide by any web provider you use , you might have a "cPanel" where these can be changed, or maybe you have to ask the provider to do the necessary changes.
Unsorted stuff
Some of the following is out of date, it refers to the original beta versions of MX not the latest release, other parts are just hints of what needs to be said ... ANY volunteers to UPDATE THIS?
It needs rewriting to be proper question and answer layout.
- There is a third party offering for helping you start and stop Cumulus MX on a Raspberry Pi. There are two other related threads initiated by same author (Jank).
- One problem various people have had is swapping from a Windows environment to a Linux environment. There are lots of new commands to learn, and specifying the interface is different so getting MX to connect to your weather station has been a problem for some.
- For setting up your web site see this page first.
- For a guide to the settings pages of MX user interface see here.
- Cumulus MX supports updating to secure web sites, this feature is not yet properly documented.
- The use of decimal comma gives rise to various problems mentioned in the forum, as the guidance above says Cumulus MX uses procedures that depend on full stops being used for the decimal point in many log files.
- There is confusion between Cumulus 1 symbols for months (mm or MM allowed) and the MX use of those symbols (mm means minutes, only MM is valid for months). If you used Cumulus 1 and set up NOAA report names using 'mm' instead of 'MM', then you have a problem in MX. Another issue raised from time to time in the forum is that if you swap from Cumulus 1 to MX it may not access the old NOAA reports if the settings in MX are set to generate different file naming to that previously used in Cumulus 1).
- Similar problems occur for those swapping from Cumulus 1 to MX if they have used any web tags that have had a date-time modifier added. Here not only do you need to be careful whether you are using the right case, but also be prepared for single modifiers on their own to have a different meaning to when that modifier appears with other modifiers.
- On Cumulus 1, some people used '.' to separate the hours and minutes (HH.nn), this causes problems in MX that only accepts a colon separator (HH:mm). Thus files like the alltime.ini might have a mix of times in different formats.
- Certain versions of Mono give problems with some locales, for example the Mono version 5 release has thrown up an issue with some locales and the short month names having an extra decimal point that Cumulus MX cannot cope with. Most of these problems are raised in the forum.
- If you want to use your Cumulus 1 data with MX, and you use decimal commas, you will need to edit the .ini files to change the decimal commas into periods/full stops. The other data files will be OK.
- If you are trying to use your Cumulus.ini file from Cumulus 1, you will need to make sure it doesn't contain any non-ASCII characters
- Cumulus MX doesn't change the fact that you can't run two programs at the same time with the same station, whatever those programs are, unless you have something in the middle handling the multiple connections (e.g. Virtual VP for Davis stations)
- If a settings screen shows red contents (red text and red boxes), invalid data is present which must be corrected before the form can be sent. One or more items on the page will have an error message.
- Cumulus MX doesn't create any image files
- Cumulus MX on Windows requires .NET 4.5 and hence will not run on the obsolete Windows XP
- The realtime interval on the internet settings screen is in milliseconds
- There is currently no 'diary' and hence also no snow recording
- The requirement to run as root/administrator may possibly be removed at some point
- When you install a new version of Cumulus MX (i.e when upgrading from one version to another), be sure to copy all of the files from the release zip file into your MX installation, and force a refresh of the interface pages in your browser. You can optionally just copy the files that I name as having changed in the announcement, but the risk is that I might have missed a changed file from the list. No settings or data are overwritten by the update.
- Some setting changes don't take effect until you restart Cumulus MX
- Note that the change to date/time formats in webtags also applies to NOAA report file formats., if you carry over your old settings
- The 'extra files' editor updates automatically when a field is changed. You can press 'enter' in a text field to get it to update, or move to another field
- The remote filename for an 'extra file' must include the path (if required) as well as the file name, just as in Cumulus 1. It does not use the 'directory' setting automatically
- If you are using your 'old' Cumulus.ini file, make sure your 'ftp site' section is [FTP site] with 'site' in lower case
- Davis stations sometime supply 'error' values such as 255 for wind speed. Cumulus ignores these and may log some of them to the diags file, if you have debug logging turned on
- Serial device names on Linux are not COM0 etc as in Windows. You need to specify something like /dev/ttyUSB0 - do a dmesg at a command line prompt and look for the actual device name from your adapter. If dmesg says your cp210x converter (as on Davis stations) is attached to ttyUSB0, for example, you need to put /dev/ttyUSB0 for the serial port name in MX. You can set the serial port name in the MX user interface, but if for some reason you want to do it by editing Cumulus.ini, note that the item in MX is called ComportName (which is a string, for the full device name, e.g. COM1 or /dev/ttyUSB0) rather than 'Port' which is just an integer. If you have not run MX yet, this item will not exist, so you will need to add it (in the [Station] section).
- If you are having problems after upgrading to Windows 10, see this post - https://cumulus.hosiene.co.uk/viewtopic.php?f=2&t=14148
- Cumulus MX will not run on Windows IOT
- The Dashboard, Now, and Gauges pages all get their data using web sockets (on port 8002). If they are not displaying data for you, you have some issue somewhere on your system preventing that from working - a firewall issue, or perhaps a browser issue. If you can't get web sockets working on your system, you could switch to using Ajax for the data. See the announcement for build 3032 for instructions.
- If you want to use a twitter.txt file and include any characters not in the 7-bit ASCII set (e.g. degree symbols or accented characters) you need to save the file as UTF-8 (without 'BOM')
- As with Cumulus 1, if you have extra web files configured, only tick 'process' for those files which have web tags in them, i.e. they need processing.
- An "error [object Object]" in your browser when you try to change settings is usually caused by using a version of Mono which does not support comma decimals, but can also be caused by an invalid setting
- Twitter have changed their authentication method, and Cumulus is now unable to perform the initial step of obtained the authentication tokens. The fix requires significant change to the code and I have no forecast for this