Process_Description:
The Cropland Data Layer (CDL) Program provides the National Agricultural Statistics Service (NASS) with internal proprietary county and state level acreage indications of major crop commodities, and secondarily provides the public with "statewide" (where available) raster, geo-referenced, categorized land cover data products after the public release of county estimates. This project builds upon the USDA's National Agricultural Statistics Service (NASS) traditional crop acreage estimation program, and integrates the enumerator collected ground survey data with satellite imagery to create an unbiased statistical estimator of crop area at the state and county level for internal use. No farmer reported data is revealed, nor can it be derived in the publicly releasable Cropland Data Layer product.
Every June thousands of farms are visited by enumerators as part of the USDA/NASS June Agricultural Survey (JAS). These farmers are asked to report the acreage, by crop, that has been planted or that they intend to plant, and the acreage they expect to harvest. Approximately 11,000 area segments are selected nationwide for the JAS. The segment size can range in size from about 1 square mile in cultivated areas to 0.1 of a square mile in urban areas, to 2-4 square miles for larger probability proportional to size (PPS) segments in rangeland areas. This division allows intensively cultivated land segments to be selected with a greater frequency than those in less intensively cultivated areas. The 150-400 square miles of ground truth collected during the JAS provides a great ground truth training set annually.
The Area Sampling Frame (ASF) is a stratification of each state into broad land use categories according to the percentage of cropland present. The ASF is stratified using visual interpretation of satellite imagery. The sampling frames are constructed by defining blocks of land whose boundaries are physical features on the ground (roads, railroads, rivers, etc). These blocks of land cover the entire state, do not overlap, and are placed in strata based on the percent of land in the block that is cultivated. The strata allow for efficient sampling of the land, as an agriculturally intensive area will be more heavily sampled than a non ag intensive area.
The enumerators draw off field boundaries onto NAPP 1:8,000 black and white aerial photos containing the segment, according to their observations and the farmer reported information. The fields are labeled and the cover type is recorded using a grease pencil on the aerial photo. Enumerators account for every field/land use type within a segment. They assign each field a cover type based upon a fixed set of land use classes for each state. Every field within a segment must fit into one of the pre-defined classes.
The program methodology is a continuous process throughout the year. The first step "Segment Preparation" establishes the training segments, digitizes the perimeters, and distributes software and data to the field offices, this goes from February to late May. Segment digitizing begins during the JAS and continues until all fields and all segments are completely digitized, this may run thru July or even until mid-October in some states depending on human resource availability. Segment cleanup analyzes the newly digitized segments with the new acquired imagery. Fields that are bad either by digitizing or cover type are corrected or removed from training. Scene processing fits each segment onto a scene by shifting, and cloud-influenced segments are removed. The cluster/classification process runs in concert with the scene processing steps, as segments are shifted they can be clustered. This process is iterative, and can run into December. Estimation can be performed once a scene is finished classification, and the user is satisfied with the outputs. Estimation can begin as early as late October and run into late January/February. The mosaic process runs once estimation is completed. It is also iterative and can go from late December to March. The mosaic for a particular state is released once the county estimates are officially released for that state.
Scene selection begins in early summer, and could run into the late fall depending on image availability. The Cropland Data Layer program primarily uses the Landsat platform for acreage estimation. However, other platforms such as Spot and the Indian IRS platforms are used to fill "data acquisition" holes within a state. A spring and summer date of observation is preferred for maximum crop cover separation for multi-temporal analysis of summer crops. If only one date of observation is available (unitemporal), a mid summer date is preferred. If only an early spring date March-May or a fall date September-October is available (unitemporal) during the growing season, then it is best to not use that scene or analysis district for estimation, as bare soil in the spring and fully senesced crops in the fall will provide erroneous results.
The clustering/classification is an iterative process, as fields get misclassified, they can be fixed or marked as bad for training and reprocessed. Known pixels are separated by cover type and clustered, within cover type using a modified ISODATA clustering algorithm, as it allows for merging and splitting of clusters. Modified implies that the output clusters are not labeled (other than as coming from the input cover type) as they can be reassigned later if desired. Clustering is done separately for each cover type (or specified combination of cover types, such as all small grains). The clustered cover types are then assembled together into one signature file, where entire scenes are classified using the maximum likelihood algorithm. Clustering is based on the LARSYS (Purdue University) ISODATA algorithm. It performs an iterative process to divide pixels into groups based on minimum variance. The pairs of clusters in close proximity (based on Swain-Fu distance) are merged. High variance clusters can be split into two clusters (variance of first principal component is used as a measure). The output of any clustering program is a statistics file which stores mean vectors and covariance matrices of final set of clusters.
The outputs are a categorized or classified image in PEDITOR format and the associated accuracy statistics for each cover type. The maximum likelihood classifier performs a pixel-by-pixel classification based on the final, combined statistics file. It calculates the probability of each pixel being from each signature; then classifies a pixel to the category with highest probability. The processing time depends on size of file to be classified (i.e. number of pixels), number of categories in the statistics file and number of input dimensions (number of bands/pixel).
For estimation purposes, clouds can be minimized by defining Analysis Districts (AD) along adjacent scene edges, by cutting the Analysis Districts by county boundary, or cutting the clouds out by primary sampling units. Analysis Districts can be individual or multiple scenes footprints that have to be observed on the same date, and analyzed as one. An AD can be comprised of one or more scenes. An AD can be defined by either a scene edge or a county boundary. Multi-temporal AD's are possible as long as both dates in all scenes are the same. A single or multi-scene AD will use all potential training fields for clustering/classification/estimation. Several factors can lead to problems in a classification, some get corrected in early edits and some do not:
Several factors can lead to problems in a classification, some get corrected in early edits and some do not: poor imagery dates, with respect to the major crops of interest, complete training fields that are incorrectly identified in the ground truth, parts of training fields that are not the same as the major crop or cover type, irrigation ditches, wooded areas, low spots filled with water, and/or bare soil areas in an otherwise vegetated field. Crops that look alike to the clustering algorithm(s) due to planting/growing cycle: spring wheat and barley at almost any time, crops in senescence, and grassy waste fields and idle cropland. Cover types that are essentially the same but used differently: wooded pasture versus woods or waste fields (only difference may be the presence of livestock), corn for grain versus corn silage, and cover crops such as rye and oats. Cover types that change signatures back and forth during the growing season: alfalfa and other hays before and after cutting, with multiple cuttings per year. Once the analyst is satisfied with the classification, the next step can be acreage estimation or image mosaicking.
Three estimation methods are available for each AD: regression, pixel ratio and direct expansion. Where available, regression is chosen as the preferred type of estimation. This approach essentially corrects the area sample (ground only) estimate based on the relationship found between reported data and classified pixels in each stratum where it is used. A regression relationship should be based on 10 or more segments for any stratum used. Where there are not enough segments in each stratum, a pixel based ratio estimator may be used which essentially combines data across stratum to get the relationship. Finally, the direct expansion (total number of possible segments times the average for sampled segment) may be used in the absence of pixel based methods. Regression adjusts the direct expansion estimate based on pixel information. It usually leads to an estimate with a much lower variance than direct expansion alone. Segments, called outliers, which do not fit the linear relationship estimated by the regression are reviewed; if errors are found, they are corrected or that segment may be removed from consideration in the analysis.
Full scene classifications (large scale) are run wherever the regression or pixel ratio estimates are usable. Estimates derived from the classification are compared to the ground data to make one final check. State estimates are made by summing pixel based estimators where available and ground data only estimators everywhere else. County estimates are then derived from the state estimates using a similar approach. Final numbers are delivered to state field offices and the NASS Agricultural Statistics Board for their use in setting the official final estimates. The states also have administrative data, such as FSA certified acres at the county level, and other NASS survey data. Every 5th year, NASS also performs the Census of Agriculture at the county level.
The Landsat TM/ETM+ scenes that SARS uses are radiometrically and systematically corrected. There is a need to tie down registration points on a continuing basis for every state in the project. Without some image/image registration, the scene registration tends to float 2-3 pixels in any given direction, for any given scene. Manual registration for every scene of every project, would be nearly impossible, as the CDL is on a repeating production cycle every year, and human resource levels for this process are low. Image recoding is necessary between different analysis districts, to rectify to a common signatures set for a state. Clouds pose a problem when trying to make acreage estimates, and there are mechanisms within Peditor to minimize their extent, as there are ways to minimize cloud coverage in the mosaic process by prioritizing scene overlap.
Each categorized scene is co-registered to EarthSat's GeoCover LC imagery (50 meters RMS), and then stitched together using Peditor's Batch program. A block correlation is run between band two from each raw scene, and band two of the ortho-base image. The registration of the GeoCover mosaicked scene and the individual raw input scenes are used to get an approximate correspondence. A correlation procedure is used on the raw Landsat scenes and the mosaicked scene to get an exact mapping of each pixel from the input Landsat scenes to the mosaicked scene. The results of the correlation are used to remap the pixels from the individual input scenes into the coordinate system of the mosaicked scene. The mosaic process now performs: 1) Precision registration of images automatically, 2) Converts each categorized image and associated statistics file to a set standard automatically (recode), 3) Specify overlap priority by scene or county, 4) Filters out clouds when possible. The scenes are stitched together using the priorities previously assigned from the scene observation dates/analysis districts map. Scenes/analysis districts with better quality observation dates are assigned a higher priority when stitching the images together. Clouds are assigned a null value on all scenes, and scenes of lower priority that are cloud free, take precedence over clouded higher priority images. Once cloud cover is established throughout the mosaic the clouds are assigned a digital value.
The Cropland Data Layer CD-ROM products contain two years (if available) of imagery in a GEOTIFF image file format. In order to maximize the visual contrast between different crops in various states, colors that provide the best contrast for the crop mix in a particular State are chosen. However, the digital values for each category within every State remain the same. So corn in ND will have the same digital number as corn in AR. See mastercat.htm on the CDL CD-ROM in the statinfo directory for a full listing by cover type.
All CDL distribution for the previous crop year is held until the release of the official NASS county estimates for the major commodities grown within a given state. Corn and Soybeans are released in March for the previous crop year - Midwestern States. Rice and Cotton are released in June for the previous crop year - Delta States. Small grains are released in March for the Great Plains States.
NASS publishes all available accuracy statistics for end-user viewing. The Percent Correct is calculated for each cover type in the ground truth, it shows how many of the total pixels were correctly classified (i.e. across all cover types). 'Commission Error' is the calculated percentage of all pixels categorized to a specific cover type that were not of that cover type in the ground truth (i.e. incorrectly categorized). CAUTION: a quoted Percent Correct for a specific cover type is worthless unless accompanied by its respective Commission Error. Example: if you classify every pixel in a scene to 'wheat', then you have a 100% correct wheat classifier (however its Commission Error is also almost 100%). The 'Kappa Statistic' is an attempt to adjust the Percent Correct using information gained from the confusion matrix for that cover type. Many remote sensing groups use the Percent Correct and/or Kappa statistics as their final measure of classification accuracy.
The NASS CDL Program is continuing efforts to reduce end-user burden, increase functionality, and take advantage of enhancements in computer technology. The Cropland Data Layer Program is a one of a kind agricultural inventory program, where every state participating in the program is re-surveyed (i.e., ground truthed) every June, and thus re-categorized. The data on the CD-ROM is in the public domain, and you are free to do with it as you choose. NASS would appreciate acknowledgment or credit regarding the source of the categorized images in any uses that you may have.
Remember, in no case is farmer reported data revealed or derivable from the public use Cropland Data Layer CD-ROM's.