Category:Files with Comma Separated Values: Difference between revisions

From Cumulus Wiki
Jump to navigationJump to search
m (add category)
m (Changes to section levels)
 
Line 10: Line 10:
'''Although these files are described as CSV format, that does not mean that the character separating one field from the next field has to be ",".''' In some locales, the comma character might separate the integer and decimal parts of a number, so the field separator character might be a semi-colon ";".
'''Although these files are described as CSV format, that does not mean that the character separating one field from the next field has to be ",".''' In some locales, the comma character might separate the integer and decimal parts of a number, so the field separator character might be a semi-colon ";".


=Quick Guide to these files=

Files using this format fall into a few classes:
# xxxxheader.txt - these files hold a list of headings for the fields of the files they are linked with, the separator character is often different e.g. it might be "|". Although you can use them to add heading when editing the associated data file in spreadsheet software, remove the headings before returning the file for use by Cumulus
# dayfile.txt - there is a single file in this class, and it has one line per day, but be aware that while the start/end of day will be [[Meteorological day|meteorological day]] for most (e.g. maximum pressure) fields i.e. new line each rollover; if that rollover is not midnight then some fields are midnight to midnight (e.g. sun hours). (You do need to be aware that Cumulus also stores data by day in [[Weather Diary]] although for legacy Cumulus 1 that diary can have several entries for a day, and for both 1 and MX there will be dates without any entry).
# Standard/Extra/Airlink - for each of these a new file is started at your rollover time on the first day of a new month, and the time interval between lines in these files depends on whether Cumulus was running at the time indicated in the line. So if the line was created from historic catch-up when Cumulus is first started, then the interval between lines is determined by how frequently the weather station retains data. If the line is created while Cumulus is running normally, then the lines are stored at the logging interval set within Cumulus.


==Editing these files==
==Editing these files==

Please see individual pages for guidance on editing pages (e.g. [[Amending dayfile]], [[Standard log files]], etc.), if you are not using the editors built into Cumulus, you may need to take a copy of the file, or stop Cumulus.



The files linked below all use plain text, so that they can be edited outside Cumulus, but there are rules about content, so be very careful about editing outside Cumulus. A "CSV editor", or a simple text editor designed for programmers, is safer to use than a spreadsheet (if you use a spreadsheet, ensure all columns are set to be "text", don't let it recognise dates, times or numbers), and you must never use a word-processor.
The files linked below all use plain text, so that they can be edited outside Cumulus, but there are rules about content, so be very careful about editing outside Cumulus. A "CSV editor", or a simple text editor designed for programmers, is safer to use than a spreadsheet (if you use a spreadsheet, ensure all columns are set to be "text", don't let it recognise dates, times or numbers), and you must never use a word-processor.


You may wish to edit into a data file a header line, to make life easier when editing values, but remember to remove that header line before saving the file.
If you are using an external editor, you may wish to edit into a data file a header line, to make life easier when editing values, but remember to remove that header line before saving the file.


The line terminator for file lines depends on the device running MX. In Microsoft Windows Operating System each line is terminated by a pair of characters (CR and LF), a UNIX-derived system (like Linux) uses just LF to end a line, and some versions of Mac Operating System use just CR to terminate each line.
The line terminator for file lines depends on the device running MX. In Microsoft Windows Operating System each line is terminated by a pair of characters (CR and LF), a UNIX-derived system (like Linux) uses just LF to end a line, and some versions of Mac Operating System use just CR to terminate each line.


If you move a file between different devices, beware the line endings might get changed (although an editor designed for programmers includes ability to select line ending).
If you move a file between different devices, beware the line endings might get changed (this is a benefit of using an editor designed for programmers as those include ability to select line ending).


Recent releases of MX are coded to recognise which line ending was previously used, and will maintain that even if you run an executable (CumulusMX.exe, CreateMissing.exe, or ExportToMySQL.exe) on another device.
Recent releases of MX are coded to recognise which line ending was previously used, and will maintain that even if you run an executable (CumulusMX.exe, CreateMissing.exe, or ExportToMySQL.exe) on another device.

=Cumulus flavour dependencies=


==Note for anyone using the Legacy Cumulus 1==
==Note for anyone using the Legacy Cumulus 1==


This Wiki page was created as part of documenting Cumulus MX, and is not therefore written for you.
'''For Legacy Cumulus 1''', only a subset of the files listed here are used. Please be aware that within those files that are used the content differs between MX and the legacy software. This Wiki category was created as part of documenting Cumulus MX, and is not therefore specifically written for you, however you will find the files you have are linked from this page.


Please note that descriptions of those files that are used by the legacy software, and tips, can be found in the Help file within the Cumulus 1 software. Appropriate parts of the Cumulus help file are linked from [[Cumulus Screenshots|individual Cumulus screens]], but it is also possible to read the whole file at [https://cumuluswiki.org/Downloads/Cumulus.chm Cumulus.chm] if your computer is using a Microsoft Windows Operating System.
'''For Legacy Cumulus 1''', only a subset of the files listed here are used. Please be aware that within those files that are used the content differs between MX and the legacy software.

Please note that descriptions of those files that are used by the legacy software, and tips, can be found in the Help file within the Cumulus 1 software, it can be read at [https://cumuluswiki.org/Downloads/Cumulus.chm Cumulus.chm] although that help file is formatted to only open if your computer is using a Microsoft Windows Operating System.


==Cumulus MX users should read this==
==Cumulus MX users should read this==


This sub-category was created specifically for Cumulus MX. It provides links to pages that give explanations for all of the Log files with a '''.txt''' extension. If you have any files with such an extension that were created by the legacy software:
This Wiki sub-category was created specifically for Cumulus MX. It provides links to pages that give explanations for all of the Log files with a '''.txt''' extension, and to a couple of other useful pages.

If you have any files with such an extension that were created by the legacy software:
# MX software normally ignores any data for dates before the first use of MX, please see [[Migrating_from_Cumulus_1_to_MX#Start_date]] for further advice
# MX software normally ignores any data for dates before the first use of MX, please see [[Migrating_from_Cumulus_1_to_MX#Start_date]] for further advice
# MX software creates more fields in each line of each of the files, than the fields created by Cumulus 1, please see [[Calculate_Missing_Values#CreateMissing.exe]] for further advice
# MX software creates more fields in each line of each of the files, than the fields created by Cumulus 1, please see [[Calculate_Missing_Values#CreateMissing.exe]] for further advice

Latest revision as of 14:27, 9 June 2022

Cumulus software uses a number of files with a .txt extension that can be described as being in comma separated value format (CSV).

Cumulus normally simply appends new lines onto the end of these files, although there is some functionality to read them for plotting graphs and for displaying/editing extreme records.

None of the files hold a header line, although the appropriate header labelling the fields can be found in a different file. (Links to those files included in list below).

Field separator

Although these files are described as CSV format, that does not mean that the character separating one field from the next field has to be ",". In some locales, the comma character might separate the integer and decimal parts of a number, so the field separator character might be a semi-colon ";".

Quick Guide to these files

Files using this format fall into a few classes:

  1. xxxxheader.txt - these files hold a list of headings for the fields of the files they are linked with, the separator character is often different e.g. it might be "|". Although you can use them to add heading when editing the associated data file in spreadsheet software, remove the headings before returning the file for use by Cumulus
  2. dayfile.txt - there is a single file in this class, and it has one line per day, but be aware that while the start/end of day will be meteorological day for most (e.g. maximum pressure) fields i.e. new line each rollover; if that rollover is not midnight then some fields are midnight to midnight (e.g. sun hours). (You do need to be aware that Cumulus also stores data by day in Weather Diary although for legacy Cumulus 1 that diary can have several entries for a day, and for both 1 and MX there will be dates without any entry).
  3. Standard/Extra/Airlink - for each of these a new file is started at your rollover time on the first day of a new month, and the time interval between lines in these files depends on whether Cumulus was running at the time indicated in the line. So if the line was created from historic catch-up when Cumulus is first started, then the interval between lines is determined by how frequently the weather station retains data. If the line is created while Cumulus is running normally, then the lines are stored at the logging interval set within Cumulus.

Editing these files

Please see individual pages for guidance on editing pages (e.g. Amending dayfile, Standard log files, etc.), if you are not using the editors built into Cumulus, you may need to take a copy of the file, or stop Cumulus.


The files linked below all use plain text, so that they can be edited outside Cumulus, but there are rules about content, so be very careful about editing outside Cumulus. A "CSV editor", or a simple text editor designed for programmers, is safer to use than a spreadsheet (if you use a spreadsheet, ensure all columns are set to be "text", don't let it recognise dates, times or numbers), and you must never use a word-processor.

If you are using an external editor, you may wish to edit into a data file a header line, to make life easier when editing values, but remember to remove that header line before saving the file.

The line terminator for file lines depends on the device running MX. In Microsoft Windows Operating System each line is terminated by a pair of characters (CR and LF), a UNIX-derived system (like Linux) uses just LF to end a line, and some versions of Mac Operating System use just CR to terminate each line.

If you move a file between different devices, beware the line endings might get changed (this is a benefit of using an editor designed for programmers as those include ability to select line ending).

Recent releases of MX are coded to recognise which line ending was previously used, and will maintain that even if you run an executable (CumulusMX.exe, CreateMissing.exe, or ExportToMySQL.exe) on another device.

Cumulus flavour dependencies

Note for anyone using the Legacy Cumulus 1

For Legacy Cumulus 1, only a subset of the files listed here are used. Please be aware that within those files that are used the content differs between MX and the legacy software. This Wiki category was created as part of documenting Cumulus MX, and is not therefore specifically written for you, however you will find the files you have are linked from this page.

Please note that descriptions of those files that are used by the legacy software, and tips, can be found in the Help file within the Cumulus 1 software. Appropriate parts of the Cumulus help file are linked from individual Cumulus screens, but it is also possible to read the whole file at Cumulus.chm if your computer is using a Microsoft Windows Operating System.

Cumulus MX users should read this

This Wiki sub-category was created specifically for Cumulus MX. It provides links to pages that give explanations for all of the Log files with a .txt extension, and to a couple of other useful pages.

If you have any files with such an extension that were created by the legacy software:

  1. MX software normally ignores any data for dates before the first use of MX, please see Migrating_from_Cumulus_1_to_MX#Start_date for further advice
  2. MX software creates more fields in each line of each of the files, than the fields created by Cumulus 1, please see Calculate_Missing_Values#CreateMissing.exe for further advice

Pages in category "Files with Comma Separated Values"

The following 12 pages are in this category, out of 12 total.