x

【地球】真棒地球观测代码

apaas.dev
2 June 2022
SEO Title
acgeospatial awesome-earthobservation-code

A curated list of awesome tools, tutorials, code, helpful projects, links, stuff about Earth Observation and Geospatial stuff!

This list was started based on #scenefromabove podcast lunchtime discussions

I have written a blog post about how this repo came into being. It includes a video of a talk I gave about it AND a podcast episode devoted to it. http://www.acgeospatial.co.uk/awesome-earthobservation-code/

Please note that this is not offically an awesome list (yet). Please help me to get it there by contributing and commenting. guidelines

Update March 2022 Still keeping an eye on updating - if you find a dead link, or new one please help us maintain. We now have over 550 links/resources! If you find a resource missing let me know and I will add, I accept PR's. Added section on Planetary Computer, COG and STAC.

Annotations are based on the headers - where available - on the github accounts

Scene From Above Podcast

Alastair Graham @ajggeoger and Andrew Cutts @map_andrew come together to present an informal podcast @eoscenefrom looking at the world of modern remote sensing and EO. Fuelled by their passion for all things raster and geospatial, the #scenefromabove podcast aims to be a mix of news, opinion, discussion and interviews.

Contents

| Earth Observation introduction |

| Open EO | remotesensing.info | Python processing | Resources for R | Languages other than Python and R | Training and Learning | Deep Learning & Machine Learning | GDAL of course | Earth Observation coding on YouTube | Google Earth Engine | Open Data Cube | Planetary Computer | QGIS and Grass | Climate & Weather resources | DEM projects | SAR | LiDAR | GEDI | InSAR | Landuse | Visualisation | EO code Competitions | ARD links | Useful EO code based twitter accounts | List of Great GitHub accounts | EO Geospatial companies or orgs making big contributions |

| Cloud Native Geospatial | STAC | COG

These sections are non EO code specific, but included to be relevant
| Interesting Non EO parts Python | Interesting Non EO parts other languages | Data | A footnote on awesome

Start Here

Earth Observation Introduction

If you are not familiar with Earth Observation then these links may help set context before you start using data. I didn't initially aim at including links like these but if you are not familiar with Earth Observation then some good resources to get you going may help prior to diving into code.

  • Earth Observation Text books - Earth Observation: Data, Processing and Applications is an Australian Earth Observation (EO) community undertaking to describe EO data, processing and applications in an Australian context and includes a wide range of local case studies to demonstrate Australia’s increasing usage of EO data.
  • ESA newcomers guide - The aim of this guide is to help non-experts in providing a starting point in the decision process for selecting an appropriate Earth Observation (EO) solution.
  • The state of satellites - The satellite systems we use to capture, analyze, and distribute data about the Earth are improving every day, creating bold new opportunities for impact in global development.

You may also wish to navigate a search of the terms satellite-imagery and earth-observation to get the latest list of topics that have these terms in their headers

Open EO

OpenEO covers many of the bases, hard to know whether to break it into different categories, it has many components. At present I mention it here at the start only.

  • Open EO - openEO develops an open API to connect R, Python, JavaScript and other clients to big Earth observation cloud back-ends in a simple and unified way.
  • openeo-processes - Interoperable processes for openEO's big Earth observation cloud processing website

Remote Sensing.info

remotesening.info warrents its own section, the vast array of tools and processing software is incredible RemoteSensing.info - Short tutorials and reference to useful software tools for the acquisition and processing of remote sensed Earth Observation data

  • RSGISLib - The Remote Sensing and GIS software library (RSGISLib) is a collection of tools for processing remote sensing and GIS datasets. The tools are accessed using Python bindings.
  • ARCSI - link currently broken
  • eodatadown - The Earth Observation Data Downloader (EODataDown) is a tool for automatically downloading and processing EO data to an analysis ready data product. This software forms a core component of a monitoring system based on EO data.
  • more to come..

Python processing of optical imagery (non deep learning)

This section full of great code and projects related to processing optical satellite imagery with Python . This section is under review Sept 2020 and being split into further categories - please suggest groupings or re assignments if needed - the idea is to make the Python code examples here easier to find. Categories are highly subjective.

Download

  • EODAG - Command line tool and a plugin-oriented Python framework for searching, aggregating results and downloading remote sensed images while offering a unified API for data access regardless of the data provider.
  • Sedas API - Python client library for the SeDAS API
  • esa_sentinel - ESA Sentinel Search & Download API
  • get_modis - Downloading MODIS data from the USGS repository Python
  • landsatexplore - Search and download Landsat scenes from EarthExplorer. Python
  • pylandsat - Search, download, and preprocess Landsat imagery Python
  • Sentinel-download - Automated download of Sentinel-2 L1C data from ESA (through wget) Python
  • sentinelsat - Search and download Copernicus Sentinel satellite images sentinelsat docs Python
  • LANDSAT-Download - Automated download of LANDSAT data from USGS website
  • Landsat-Util - A utility to search, download and process Landsat 8 satellite imagery Python
  • sentineleof Searches for and downloads the precise/restituted orbit files for all Sentinel-1 SAFE products in a given directory (using ESA's new API). Python
  • Sentinel-1_POE_orbit_download - Automatically download Sentinel-1 POE orbit data with a given product list. Python
  • data-prep-scripts - This collection of R and Python scripts can be used to download data and perform basic data processing functions such as georeferencing, reprojecting, converting, and reformatting data. All scripts are available for download from the LP DAAC User Resources BitBucket Code Repository.
  • Stream NASA data directly into Python objects - Skip the download! Stream NASA data directly into Python objects from blog post
  • sat-extractor - Extract Satellite Imagery from public constellations at scale Python

Processing imagery - post processing

  • StarFM for Python - The STARFM fusion model for Python (image fusion)

  • Remote Sensing indicies calc - Calculate spectral remote sensing indices from satellite imagery

  • EarthPy - A package built to support working with spatial data using open source python. docs

  • RasterFrames / pyrasterframes - brings together Earth-observation (EO) data access, cloud computing, and DataFrame-based data science. docs

  • SIF tools - some tools for accessing OCO-2 data

  • SIAC - A sensor invariant Atmospheric Correction (SIAC) alg doc

  • S2_TOA_TO_LAI - From Sentinel 2 TOA reflectance to LAI

  • cresi - Road network extraction from satellite imagery, with speed and travel time estmates

  • 6S_emulator - Atmospheric correction in Python using a 6S emulator

  • bv - Quickly view satellite imagery, hyperspectral imagery, and machine learning image outputs directly in your iTerm2 terminal. Python

  • mapchete - Tile-based geodata processing using rasterio & Fiona Python

  • unmixing - Interactive tools for spectral mixture analysis of multispectral raster data in Python

  • landsat and sentinel fusion - Complementarity Between Sentinel-1 and Landsat 8 Imagery for Built-Up Mapping in Sub-Saharan Africa Python

  • Planet Movement - Find and process Planet image pairs to highlight object movement. Python

  • cedar-datacube - cedar - Create Earth engine Datacubes of Analytical Readiness Python docs

  • stems - Spatio-temporal Tools for Earth Monitoring Science - Spatio-temporal Tools for Earth Monitoring Science Python docs

  • ipyearth - An IPython Widget for Earth Maps Python

  • Python-for-remote-sensing - Python codes for remote sensing applications will be uploaded. blog

  • esda dissertation - MSc Energy Systems & Data Analytics dissertation project notebooks - identifying solar PV from aerial imagery with computer vision Python

  • geff_notebooks - Jupyter notebooks to post-process fire danger data using Python/xarray

  • river-width - Extracts water features from 4 band NAIP imagery and calculates river metrics. Python

  • get_river_width - Find the river width (and other properties) from a masked water image Python

  • extract_water - Extract water from nIR imagery Python

  • pyresample - Geospatial image resampling in Python

  • spatialist - A Python module for spatial data handling

  • CometTS - Comet Time Series Toolset for working with a time-series of remote sensing imagery and user defined polygons

  • Telluric - telluric is a Python library to manage vector and raster geospatial data in an interactive and easy way

  • onearth - High-performance web services for tiled raster imagery and vector tiles Python

  • geocube - Tool to convert geopandas vector data into rasterized xarray data. Python docs

  • Opensource-OBIA_processing_chain - An open-source semi-automated processing chain for urban OBIA classification. Grass Python

  • verde - Processing and gridding spatial data using Green's functions

  • s2p - Satellite Stereo Pipeline Python

  • xcube - xcube is a Python package for generating and exploiting data cubes powered by xarray, dask, and zarr

  • geonotebook - A Jupyter notebook extension for geospatial visualization and analysis Python

  • tatortot - Prototype for a simple image annotation tool Python

  • tiletanic - Python library to support generalized geographic tiling schemes

  • Intro to Python GIS - Great free 3-day course by the University of Helsinki on GIS processing with Python

  • openaq-s5 - Map openaq data onto Sentinel5P data using AWS lambda

  • vegetation health - Predicting vegetation health from precipitation and temperature

  • Satellite-Image-Analysis - PlanetScope, Landsat-8 and Sentinel-2 Image analysis Python codes

  • felicette - Satellite imagery for dummies. Python

  • CostalSat - Global shoreline mapping tool from satellite imagery Python

  • Python-Remote-Sensing-Scripts - Python 3. X scripts for remote sensing processing

  • fc-up42 - UP42 Block for Fractional Cover calculation from Sentinel 2 L2A Data Python

  • Opensource_OBIA_processing_chain - An open-source semi-automated processing chain for urban OBIA classification.

  • nansat - Scientist friendly Python toolbox for processing 2D satellite Earth observation data. Pythondocs

  • nansat-lite - nansat-lite is not a full nansat build for Python 3.5. Only bits of code from main classes, to start with. Eventually, if need it, more code will be added.

  • IEO - Irish Earth Observation (IEO) remote sensing data processing Python module Python

  • IEOtools - Tools for managing Earth observation data. Currently only supports Landsat imagery Python

  • xcube-gen and xcube-geodb - In this Notebook we present how to, Access data via xcube-sh with a short excursion to the xcube Generator User Interface Use xcube-geodb.

  • pykic - 'Python' module for remote sensing and GIS domain (image/signal, vector, miscellaneous processing)

  • ukis-csmask - masks clouds and cloud shadows in Sentinel-2, Landsat-8, Landsat-7 and Landsat-5 images Python

  • jeolib-pyjeo - pyjeo is a library for image processing for geospatial data implemented in JRC Ispra. Python

  • pyrgis - This repository cointains the source code of the 'pyrsgis' Python package.

  • EOReader - Opensource Python library reading optical and SAR sensors, loading and stacking bands in a sensor-agnostic way.

  • LandSurfaceClustering - Land surface classification using remote sensing data with unsupervised machine learning (k-means) Python

Cloud Native Geospatial

  • aws-sat-api-py - Process Satellite data using AWS Lambda functions
  • GeoLambda - Create and deploy Geospatial AWS Lambda functions Python
  • rio-viz - Visualize Cloud Optimized GeoTIFF in browser html Python
  • Sentinel-s3 - Python libraries for extracting Sentinel-2's metadata from Amazon S3

STAC

  • stac-utils - Tools for working with SpatioTemporal Asset Catalogs (STAC) (perhaps worth going here first for STAC) Python Javascript
    • pystac - Python library for working with any SpatioTemporal Asset Catalog (STAC)
      • stactools - Command line utility and Python library for STAC
      • pystac-client - Python client for STAC Catalogs and APIs
    • pgstac - Schema, functions and a Python library for storing and accessing STAC collections and items in PostgreSQL
    • stac-fastapi - STAC API implementation with FastAPI. Python
    • STAC Spec - SpatioTemporal Asset Catalog specification - making geospatial assets openly searchable and crawlable
    • stac-validator - Validator for the stac-spec Python
    • stackstac - Turn a list of STAC items into a 4D xarray DataArray Python
    • stac-nb - STAC in Jupyter Notebooks Python
    • qgis-stac-plugin - QGIS plugin for reading STAC APIs Python
    • easystac - A Python package for simple STAC queries

COG

  • COG Validator - Cloud Optimized GeoTIFF validation service
  • titiler - A modern dynamic tile server built on top of FastAPI and Rasterio/GDAL.
  • cogeo-mosaic - Create and use COG mosaic based on mosaicJSON Python
  • Sentinel-2-cog - Convert Sentinel-2 JPEG 2000 to COG with AWS Lambda Python
  • COG Dumper - Dumps tiles out of a cloud optimized geotiff Python
  • async-cog-reader - Read Cloud Optimized GeoTiffs without GDALPython
  • aiocogeo - Asynchronous cogeotiff reader Python
  • cogeotiff - High performance cloud optimised geotiff reader
  • ecw-converter - Dockerised Python scripts & Nextflow pipeline for converting ecw files to either geotiffs or Cloud Optimised Geotiffs (COGs)
  • COG pptx/pdf - talk on COG
  • gocog - A Cloud Optimised GeoTIFF reader written in Go

Case studies / Projects

  • Povetry predition using satellite imagery - Poverty Prediction by Combination of Satellite Imagery
  • Python from space - Python Examples for Remote Sensing
  • count blue pixels - This project is an experiment in using simple image processing techniques on satellite images downloaded from Google Maps in order to quantify the relative density of temporary shelters in adjacent qudarants. Python Ruby
  • Satellite imagery analysis with Python - Getting acquainted with the concept of satellite imagery data and how it can be analyzed to investigate real-world environmental and humanitarian challenges. Python Jupyter Notebooks associated blog
  • Satellite imagery in Pakistan - This repository contains a study how we can examine the vegetation cover of a region with the help of satellite data. The notebook in this repository aims to familiarise with the concept of satellite imagery data and how it can be analyzed to investigate real-world environmental and humanitarian challenges.
  • SentinelBot - A twitter bot which processes raw sentinel data Python SentinelBot on twitter
  • ap-latem - Detection of slums and informal settlements from satellite imagery Python
  • local_structire_wpb-severity - Analysis of drone imagery to characterize forest structure and severity of a tree killing insect Python
  • Truck_Detection_Sentinel2_COVID19 - This repository is designated to detecting trucks using Sentinel-2 data. Python

Company specific examples

(you may need to create an account to use these resources)

  • Planet notebooks - interactive notebooks from Planet Engineering Python
  • Planet-client-API - Python client for Planet APIs
  • Maxar GDBx tools - Python SDK for using GBDX.
  • gdbx-surface-water - Reservoir surface area detection with Digital Globe imagery and Bayesian methods
  • SentinelHub-py - Download and process satellite imagery in Python using Sentinel Hub services.
  • sentinel2-cloud-detector - Sentinel Hub Cloud Detector for Sentinel-2 images in Python
  • Orbit predictor - Python library to propagate satellite orbits.
  • up42-py - Python SDK for UP42, the geospatial marketplace and developer platform. Python
  • S2-superresolution - Deep Learning-based algorithm to upsample all Sentinel-2 bands to 10m. Also an example how to use GPUs on UP42. Python
  • icecube - Create time-series datacubes for supervised machine learning with ICEYE SAR images. Python

Reflectance / pre processing

  • Landsat7 errors - Identifies errors in raw values of Landsat 7
  • PyProSail - Python interface to the ProSAIL leaf/canopy reflectance model
  • Py6S - A Pythoninterface to the 6S Radiative Transfer Model
  • prosail - Python bindings for the PROSAIL canopy reflectance model
  • ACOLITE_MR - ACOLITE_MR: Atmospheric correction for aquatic applications of metre-scale satellites
  • radiometric_normalization - Implementation of radiometric normalization workflows Python
  • color_balance - Balance your colors! Python
  • data-retrieval-in-EO - data-retrieval-in-EO, a project with reports from TU wien

Python libraries related to EO

  • rasterio - Rasterio reads and writes geospatial raster datasets
  • Xarray pyconuk 2018 - Code and slides for my talk at PyCon UK 2018 on XArray Python
  • RasterStats - Summary statistics of geospatial raster datasets based on vector geometries. Python
  • SatPy - Python package for earth-observing satellite data processing
  • pyimpute - Spatial classification and regression using Scikit-learn and Rasterio Python
  • dask-rasterio - Read and write rasters in parallel using Rasterio and Dask Python
  • rioxarray - geospatial xarray extension powered by rasterio docs
  • xarray-spatial - Raster-based Spatial Analysis for Python
  • actinia core - Actinia Core is an open source REST API for scalable, distributed, high performance processing of geographical data that uses mainly GRASS GIS for computational tasks. Python
  • actinia satellite plugin - This actinia plugin is designed for efficient satellite data handling, especially Landsat and Sentinel-2 scenes Python
  • Whitebox Python - WhiteboxTools Python Frontend
  • ukis-pysat - generic classes and functions to query, access and process multi-spectral and SAR satellite images

Testing your code

  • image-similarity-measures - Implementation of eight evaluation metrics to access the similarity between two images. Python
  • fake-geo-images - A module to programmatically create geotiff images which can be used for unit tests. Python

Resources for R

R is not my area of expertise so this section is lighter than I'd like, plus I'd love to know what is a useful resource Books! Geospatial R Books - some R books on geospatial

  • R-Spatial - This book provides a short introduction to satellite data analysis with R.

  • GDAL Cubes - Earth Observation Data Cubes from Satellite Image Collections. Also here on github

  • Image Classification with RandomForests in R - The goal of this post is to demonstrate the ability of R to classify multispectral imagery using RandomForests algorithms.

  • R code for ML in Sat imagery - # Random Forest image classification Adapted from stackoverflow.

  • whiteboxR - An R frontend of the advanced geospatial data analysis platform - whitebox-tools.

  • RasterVIS - Methods for enhanced visualization and interaction with raster data. It implements visualization methods for quantitative data and categorical data, both for univariate and multivariate rasters. It also provides methods to display spatiotemporal rasters, and vector fields.

  • Landsat - Processing of Landsat or other multispectral satellite imagery. Includes relative normalization, image-based radiometric correction, and topographic correction options.

  • rnoaa - R interface to many NOAA data APIs

  • MODISTools - Interface to the MODIS Land Products Subsets Web Services Docs

  • A Step-by-Step Guide to Making 3D Maps with Satellite Imagery in R - Walk you through [on] how to obtain the data required to make these types of maps, as well as the R code used to generate them

  • landsatlinkr - An automated system for creating spectrally consistent and cloud-free Landsat image time series stacks from a combination of MSS, TM, ETM+, and OLI sensors project

  • planetR - (early development) R tools to search, activate and download satellite imagery from the Planet API

  • ForestTools - Detect and segment individual tree from remotely sensed data

  • lidR - R package for airborne LiDAR data manipulation and visualisation for forestry application. Plus lidRplugins - Extra functions and algorithms for lidR package

  • Spatiotemporal Arrays: Raster and Vector Datacubes - Spatiotemporal Arrays, Raster and Vector Data Cube

  • getSpatialData - An R package making it easy to query, preview, download and preprocess multiple kinds of spatial data docs

  • RStoolbox - RStoolbox is a R package providing a wide range of tools for your every-day remote sensing processing needs.

  • rHarmonics - R package for harmonic modelling of time-series data

  • rerddap - R client for working with ERDDAP servers docs reference the ERDDAP Server

  • Spatial_Data_in_R - SWIRL-course on spatial data in R

  • cognition-datasources - Standardized query interface for searching geospatial assets via STAC.

  • caliver - caliver: CALIbration and VERification of gridded fire danger models R

  • clip_time_series - create snippets of Landsat and Sentinel imagery

  • RGISTools - Tools for Downloading, Customizing, and Processing Time Series of Satellite Images from Landsat, MODIS, and Sentinel

  • Grassland-Species-Classification - Codes for: Javier Lopatin, Fabian E. Fassnacht, Teja Kattenborn, Sebastian Schmidtlein. Mapping plant species in mixed grassland communities using close range imaging spectroscopy. Remote Sensing of Environment 201, 12-23. R

  • UAV-InvasiveSpp - Mapping invasive tree species in Chile using UAV R

  • Peatland-carbon-stock - Codes for: Lopatin, J., et al. (2019). Using aboveground vegetation attributes as proxies for mapping peatland belowground carbon stocks. Remote Sens. Environ. 231, 111217 R

  • SpeciesRichness-GLMvsRF-LiDAR - R-codes for: Lopatin, J., Dolos, K., Hernández, J., Galleguillos, M., Fassnacht, F. E. (2016): Comparing Generalized Linear Models and random forest to model vascular plant species richness using LiDAR data in a natural forest in central Chile. Remote Sensing of Environment 173, pp. 200–210. 10.1016/j.rse.2015.11.029

  • tree_segmentation - LiDAR tree segmentation R

  • swdt - Sentinel-1 Water Dynamics Toolkit R

  • What_are_data_cubes - Analyzing and visualising spatial and spatiotemporal data cubes - Part I

  • classifying_satellite_imagery_in_R - For this tutorial, we use Landsat 8 imagery from Calgary

  • planetR - R tools to search, activate and download satellite imagery from the Planet API.

Languages other than Python and R

  • Georust - A collection of geospatial tools and libraries written in Rust
  • ArchGDAL - Julia - Julia A high level API for GDAL - Geospatial Data Abstract
  • Julia_Geospatial - Examples for a blog series on Geospatial Julia using ArchGDAL
  • GeoTrellis homepage - GeoTrellis is a geographic data processing engine for high performance applications. Scala
  • GDAL with GoLang - Go (golang) wrapper for GDAL, the Geospatial Data Abstraction Library
  • C++ gdalcubes - Earth observation data cubes from GDAL image collections C++
  • RSGLib - The remote sensing and GIS software library (RSGISLib) is a set of C++ libraries and commands for the processing of spatial data (raster and vector). Functionality is via Python interface though
  • WhiteBox with Java - An open-source GIS and remote sensing package - Java
  • Perl extension for GDAL - Geo:: GDAL - Perl extension for the GDAL library for geospatial data
  • PDAL - PDAL is Point Data Abstraction Library. GDAL for point cloud data.
  • force - Framework for Operational Radiometric Correction for Environmental monitoring in c
  • LLR-landTrendr - Landsat-based Detection of Trends in Disturbance and Recovery algorimth modified to accept LandsatLinkr-processed imagery. IDL
  • Global Forest Watch - Global Forest Watch: An online, global, near-real time forest monitoring tool
  • conda recipes - Conda recipes for remote sensing Shell
  • Landsat-solar-elevation - A web app that plots annual solar elevation at the time of Landsat overpass for locations throughout the earth JavaScript
  • staccato - Java implementation of the STAC spec
  • stac4s -a scala library with primitives to build applications using the SpatioTemporal Asset Catalogs specification
  • stac-browser - A Vue-based STAC browser intended for static + dynamic deployment
  • EO Browser Custom Scripts - A repository of custom scripts to be used with Sentinel Hub JavaScript
  • sentinelhub-js - Download and process satellite imagery in JavaScript or TypeScript using Sentinel Hub services.
  • s3tbx - A toolbox for the OLCI and SLSTR instruments on board of ESA's Sentinel-3 satellite - Java
  • s2tbx - Sentinel 2 Toolbox (s2tbx) - Java
  • s1tbx - The Sentinel-1 Toolbox - Java
  • snap_engine - ESA Earth Observation Toolbox and Java Development Platform
  • Worldview - Interactive interface for browsing global, full-resolution satellite imagery Javascript application here
  • Orfeo ToolBox (OTB)- An open-source project for state-of-the-art remote sensing, including a fast image viewer, apps callable from Bash, Python or QGIS, and a powerful C++ API.
  • landsat_preprocess - IPython notebook documenting a workflow for preprocessing Landsat data Shell
  • stac-mode-validator - Simple proof-of-concept to validate STAC Items, Catalogs, Collections and core extensions with node. JavaScript
  • aiforearth-landcover-app - web mapping app to test, tweak and train the land cover classification from a deep neural network model
  • tiffhax - tiff metadata hex viewer Go
  • Fmask - The software called Fmask (Function of mask) is used for automated clouds, cloud shadows, and snow masking for Landsats 4-8 and Sentinel 2 images. Matlab
  • resto - A metadata catalog and search engine for geospatialized data PHP Stac!
  • pktools - pktools is a suite of utilities written in C++ for image processing with a focus on remote sensing applications. It relies on the Geospatial Data Abstraction Library (GDAL) and OGR.

Training and learning

  • Earth Data Lab - A site dedicated to tutorials, course and other learning materials and resources developed by the Earth Lab team
  • EO College Github
  • profLewis-geog0111 - UCL Geography: 4th year course, Scientific Computing
  • Intro to Geospatial Vector and Raster - Data Carpentry’s aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain.
  • Andrew Cutts Github - I am an Earth Observation and Geospatial enthusiast, primarily using Python to automate and process images at scale using computer vision
  • Open Geo Tutorial V2 - Tutorial of fundamental remote sensing and GIS methodologies using open source software in Python
  • Open Geo Tutorial V1 - Tutorial of basic remote sensing and GIS methodologies using open source software (GDAL in Python or R)
  • Foss4gUKJupyter - FOSS4G UK 2019 Workshop "Geoprocessing with Jupyter Notebooks"
  • Geoprocessing with Python - GIS circa 2009 - This material is really old and some of it is outdated (not all, though!). One of these days I might get around to putting newer class materials online, but you're stuck with this for now.
  • training-workshop - This repo contains all materials used on Planet's training workshop for Bahrain Defense Force
  • sentinel - Repository created for the Earth Observation Sentinel project (use with SNAP) Python
  • Python for Geospatial Analysis - A crashcourse introduction to using Python to wrangle, plot, and model geospatial data Python

Deep learning and Machine Learning

Curated lists

Labelling

Specific examples

  • TernausNetV2 - TernausNetV2: Fully Convolutional Network for Instance Segmentation (paper)
  • CNN-Sentinel -Analyzing Sentinel-2 satellite data in Python with Keras (repository of our talks at Minds Mastering Machines 2019 and PyCon 2018)
  • Image patches - Using a U-Net for image segmentation, blending predicted patches smoothly is a must to please the human eye.
  • Fast AI Satellite imagery resources
  • Crop yield prediction - The motive here is to predict the yield of crops of a particular farm by the change in pixels of the image of farm yearly. Uses Tensorflow
  • Houston Flooding with deep learning - Using A Segmentation Neural Net to map out flooded areas of Houston TX using satellite imagery
  • Satellite Imagery Classification with R - Pixel based classification of satellite imagery - feature generation using Orfeo Toolbox, feature selection using Learning Vector Quantization, CLassification using Decision Tree, Neural Networks, Random Forests, KNN and Naive Bayes Classifier
  • SpaceNet building detection - Project to train/test convolutional neural networks to extract buildings from SpaceNet satellite imageries.
  • Road segmentation - Road Detection in satellite imagery. Semantic segmentation is the process of classifying each pixel of an image into distinct classes using deep learning. This aids in identifying regions in an image where certain objects reside. This aim of this project is to identify and segment roads in aerial imagery. Detecting roads can be an important factor in predicting further development of cities, and this concept plays a major role in GeoArchitect (A project which I started). Segmentation of roads is important to map-based applications and is used for finding distances or shortest routes between two places.
  • Super resolution (srcnn) - Super Resolution for Satellite Imagery
  • Pixel decoder - A tool of running deep learning algorithms for semantic segmentation with satellite imagery
  • Detecting ships - Using Satellite Imagery to detect ships (Basic Object Detection)
  • deepOSM - Train a deep learning net with OpenStreetMap features and satellite imagery.
  • Keras for computer vision (Maxime Lenormand GitHub) - Introductions to Keras to perform computer vision tasks, with data exploration, error analysis and improving results.
  • Airplane image classification - This article details building a ML pipeline to classify the presence of planes in satellite images using a Convolutional Neural Network (CNN).
  • TorchSat - an open-source deep learning framework for satellite imagery analysis based on PyTorch. Python docs
  • ml_drought - Machine learning to better predict and understand drought Python. docs
  • pycrop yield prediction - A PyTorch Implementation of Jiaxuan You's Deep Gaussian Process for Crop Yield Prediction Python
  • neat-EO - Efficient AI4EO OpenSource framework Python
  • dfc2020_baseline - Simple Baseline for the IEEE GRSS Data Fusion Contest 2020 Python
  • Planesnet - Labeled training data for detection of aircraft in Planet satellite imagery
  • Planesnet detector - Detect airplanes in Planet imagery using machine learning
  • shipsnet - Detect container ships in Planet imagery using machine learning
  • Deep Learning for satellite imagery - Deep learning courses and projects
  • DeepNetsForEO - Deep networks for Earth Observation
  • mlhub-tutorials - Tutorials to access Radiant MLHub Training Datasets Python mlhub
  • EO Learn - Earth observation processing framework for machine learning in Python
  • LabelMaker - Data Preparation for Satellite Machine Learning docs Python
  • Solaris - CosmiQ Works Geospatial Machine Learning Analysis Toolkit docs
  • SpaceNet6 Baseline - Baseline algorithm for the SpaceNet 6 Challenge
  • Robosat - Semantic segmentation on aerial and satellite imagery. Extracts features such as: buildings, parking lots, roads, water, clouds
  • EO flow - Collection of TensorFlow 2.0 code for Earth Observation applications
  • Azavea - RasterVision - An open source framework for deep learning on satellite and aerial imagery.
  • raster-vision-aws - A CloudFormation template for deploying Raster Vision Batch jobs to AWS.
  • TensorBoard with sat imagery - training land cover segmentation models with high resolution satellite imagery and how to use TensorBoard to create a visual understanding of model training.
  • predicting_poverty - Combining satellite imagery and machine learning to predict poverty website
  • satellite led liverpool - Data and code for the paper "Remote Sensing-Based Measurement of Living Environment Deprivation - Improving Classical Approaches with Machine Learning", by Dani Arribas-Bel, Jorge Patiño and Juanca Duque
  • pixel_level_land_classification - Tutorial demonstrating how to create a semantic segmentation (pixel-level classification) model to predict land cover from aerial imagery. This model can be used to identify newly developed or flooded land. Uses ground-truth labels and processed NAIP imagery provided by the Chesapeake Conservancy.
  • satellite-image-object-detection - YOLO/YOLOv2 inspired deep network for object detection on satellite images (Tensorflow, Numpy, Pandas). Python
  • contrastive_sensor_fusion - Open-source code for the paper "Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach ". Python
  • ai4eo - Python routines for Machine Learning applications for Earth Observation
  • NGVEO - Deep learning for Earth Observation Python
  • iris - Semi-automatic tool for manual segmentation of multi-spectral and geo-spatial imagery.
  • ESRCNN-for-Landsat8-Sentinel2-Fusion - ESRCNN-for-Landsat8-Sentinel2-Fusion
  • urban-environments - Code for constructing the urban environments dataset and for land use classification via convolutional neural networks Python
  • AIforEarth-API-Development - This is an API Framework for AI models to be hosted locally or on the AI for Earth API Platform Python
  • ai4eutils - Shared utility scripts for AI for Earth projects and team members Python
  • odeon-landcover - ODEON stands for Object Delineation on Earth Observations with Neural network. It is a set of command-line tools performing semantic segmentation on remote sensing images (aerial and/or satellite) with as many layers as you wish Python. You may need to inspect development branches to learn more.
  • SAR2NDVI_CNN - A CNN is trained to perform the estimation of the NDVI, using coupled Sentinel-1 and Sentinel-2 time-series. Python
  • Museo Toolbox - A library to simplify the use of raster/vector, especially for machine learning and remote sensing. Python
  • TorchGeo - Extension of PyTorch that includes datasets, transforms, and models for geospatial data. Python
  • simrdwn - CosmiQ Works rapid satellite imagery object detection
  • crop-mask - End-to-end workflow for generating high resolution cropland maps Python
  • road_detection_mtl - Road Detection from Remote Sensing Imagery using Deep Learning Techniques Python

GDAL of course

Earth Observation coding on YouTube

(presenters listed where possible)
There are many videos relating to Earth Observation and coding, especially Python. This is really such a small collection of videos here. I have attempted to only include ones with good audio and code examples.

  • xArray at PyConUK2018 - Robin Wilson - Processing thousands of satellite images to understand air quality in the UK - it's efficient and easy with XArray
  • Visualizing & Analyzing Earth Science Data Using PyViz & PyData - Julia Signell - In this talk, we'll work through some specific workflows and explore how various tools - such as Intake, Dask, Xarray, and Datashader - can be used to effectively analyze and visualize these data. Working from within the notebook, we'll iteratively build a product that is interactive, scalable, and deployable.
  • Hands on Satellite Imagery 2019 edition - Sara Safavi - In this tutorial, gain hands-on experience exploring Planet’s publicly-available satellite imagery and using Python tools for geospatial and time-series analysis of medium- and high-resolution imagery data. Using free & open source libraries, learn how to perform foundational imagery analysis techniques and apply these techniques to real satellite data.
  • Python from space - Katherine Scott - In this talk we will work through a jupyter notebook that covers the satellite data ecosystem and the python tools that can be used to sift through and analyze that data. Topics include python tools for using Open Street Maps data, the Geospatial Data Abstraction Library (GDAL), and OpenCV and NumPy for image processing.
  • Remote Sening with Python in Jupyter - In this video we're looking at using Google Earth Engine in Jupyter with the Python API.
  • Writing Image Processing Algorithms with ArcGIS/ArcPy - Jamie Drisdelle - learn how your algorithms can integrate with the raster processing and visualization pipelines in ArcGIS. We’ll demonstrate the concept and discuss the API by diving deep into a few interesting examples with a special focus on multidimensional scientific rasters.
  • Google Earth Engine Python - Qiusheng Wu - Introducing the geemap Python package for interactive mapping with Google Earth Engine and ipyleaflet.
  • Google Earth Engine EE101 Condensed - Noel Gorelick - Introduction to the Earth Engine API and a conceptual overview of key functionality such as compositing, reducing, mapping, zonal statistics and cluminating with building a small app.
  • Image classification with RandomForests using the R languageIn this video I show how to import a Landsat image into R and how to extract pixel data to train and fit a RandomForests model. I also explain how to conduct image classification and how to speed it up through parallel processing.
  • GeoPython 2019 stream - 17:23 Machine Learning for Land Use/Landcover Statistics of Switzerland (Adrian Meyer), 50:58 How to structure geodata, 1:18:13 Terrain segmentation with label bootstrapping for lidar datasets, case of doline detection (Rok Mihevc), 2:34:41 Bias in machine learning, 3:06:23 Software for planning research aircraft missions (Reimar Bauer), 3:32:38 How Technology Moves Fast (PJ Hagerty) , 5:02:05 Spotting Sharks with the TensorFlow Object Detection API (Andrew Carter), 5:40:23 Center for Open Source Data and AI Technologies (CODAIT), 6:03:40 Bayesian modeling with spatial data using PyMC3 (Shreya Khurana) (Sound at 6:04:23 ^^), 7:02:45 Understanding and Implementing Generative Adversarial Networks(GANs) (Anmol Krishan Sachdeva), 7:37:00 Messaging with Satellites from Anywhere on the Planet (Andrew Carter), 8:04:52 Automation of the definition and optimizatino of census sampling areas using AREA (GRID3) (Freja Hunt), 8:35:26 Coastline Mapping with Python, Satellite Imagery and Computer Vision (Rachel Keay)
  • Google Earth Engine in QGIS - This playlist looks at the GEE plugin for QGIS
  • Handling and analysing vector and raster data cubes with R - Edzer Pebesma (Institute for Geoinformatics, University of Münster) Summary: vector and raster data cubes include vector and raster data as special cases, but extend this to vector time series, OD matrices, multi-band raster data, multi-band raster time series, multi-attribute vector or raster time series, and more general to array data where one ore more dimensions are associated with space and/or with time. Examples come from pretty much all areas dealing with spatiotemporal data. This tutorial will go through a large number of examples to illustrate this idea, mostly focusing on the packages stars and sf and those supporting their classes (like tmap, mapview, gstat, ggplot2).
  • QiushengWu's youtube - This youtube channel has pretty much everything you need Earth Engine, git, colab, Python, Geoscience. Highest quality stuff.
  • The OpenDataCube Conference 2021 - Playlist from the 2021 conference
  • Dask and Geopandas - Scalable geospatial data analysis with Dask| Dask Summit 2021

Earth Engine

JavaScript & Python & R

Best to start here Awesome_GEE - A curated list of Google Earth Engine resources.

Open Data Cube

  • Opendatacube
    • Datacube Core - The Open Data Cube (ODC) makes analysing continental scale Earth Observation data through time easier
    • Datacube Notebooks - Extra documentation about using the ODC with Jupyter Notebooks
    • Datacube Explorer - An ODC metadata browser that includes a STAC API and extent visualisations
    • ODC Tools - a suite of Python apps and libraries that help you use the ODC or EO data in a wide range of applications
  • Digital Earth Australia Notebooks - Repository for Jupyter Notebooks, tools and workflows for continental-scale earth observation/geospatial analysis with Open Data Cube and xarray Python
  • Digital Earth Africa Notebooks - Repository for Jupyter Notebooks, tools and workflows for continental-scale earth observation/geospatial analysis with the Open Data Cube and xarray Python, this time with a focus on Africa
  • Google Earth Engine Python examples - Various examples for Google Earth Engine in Python using Jupyter Notebook

Planetary Computer

** new and under dev **

QGIS and Grass

Climate and weather based resources

These are Python resources. Please see R resources for info on R

  • s3 tools - A collection of sentinel 3 processing tools Python
  • eumetsat -python - Shows how to read and plot satellite data from EUMETSAT NETCDF files Python
  • unidata on GOES-16 - This notebook shows how to make a true color image from the GOES-16 Advanced Baseline Imager (ABI) level 2 data. We will plot the image with matplotlib and Cartopy.Python
  • MetPy - MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data. Python
    • MetPy docsPython
    • aqua-monitor - Monitoring surface water changes from space at global scale. Also checkout the app Python
  • Ocean Color - Modis - introduction to accessing and plotting ocean color satellite data from MODIS Python
  • Climate data science - Climate Data Science and Earth Observation with Python
  • COST-EUMETSAT-Training - Material, data and presentations for the COST-EUMETSAT training school
  • eumetsat - Tools for downloading and processing satellite images from EUMETSAT
  • coda_eumetsat - Coda Eumetsat (coda.eumetsat.int) client for downloading data
  • ai4eo-forecast - Developing an open source library to compare Earth Observation and weather forecast services with the actual measurements and assess the accuracy of the forescast Python

EUMETlab

Such a vast collection of resources that it warrants a sub section within Climate and weather based resources

  • EUMETlab - This page contains groups of code repositories that have been made open to the public by EUMETSAT and our collaborators.
    • atmosphere - LTPy - Learning tool for Python on Atmospheric Composition Data is a Python-based training course on Atmospheric Composition Data. The training course covers modules on data access, handling and processing, visualisation as well as case studies.
    • sentinel-downloader - Python-based Sentinel satellite data downloader. This script allows for batch downloading of Sentinel data selected by various criteria include date, location, sensor, child products, flags and more.
    • olci-iop-processor - Code to produce Inherent Optical Properties from Level-2 OLCI data. You can find out more about the associated study here

DEM projects

  • Tin Terrain - A command-line tool for converting heightmaps in GeoTIFF format into tiled optimized meshes.
  • TauDEM - Terrain Analysis Using Digital Elevation Models (TauDEM) software for hydrologic terrain analysis and channel network extraction. Docs
  • DEM.net - Digital Elevation model library in C#. 3D terrain models, line/point Elevations, intervisibility reports. Docs
  • Stereo Mapping to create Elevation with Python - Satellite Stereo Pipeline
  • DSM2DTM - Code for the paper - Comparison of Digital Building Height Models Extracted from AW3D, TanDEM-X, ASTER, and SRTM Digital Surface Models over Yangon City Python
  • The Stereo Pipeline (NASA) - The NASA Ames Stereo Pipeline (ASP) is a suite of free and open source automated geodesy and stereogrammetry tools designed for processing stereo imagery captured from satellites

SAR

  • SAR docker - Source files for Docker image mort/sardocker/
  • awesome SAR - A curated list of awesome Synthetic Aperture Radar (SAR) software, libraries, and resources.
  • pyroSAR - framework for large-scale SAR satellite data processing
  • PyRAT - General purpose Synthetic Aperture Radar (SAR) postprocessing software package Python
  • RITSAR - Synthetic Aperture Radar (SAR) Image Processing Toolbox for Python
  • PySAR - PyAR is a perpetually incomplete, general-purpose toolbox for common post-processing tasks involving synthetic aperture radar (SAR).Python C++
  • sarbian - a plug’n play Operation System (based on Debian Linux) with all the freely and openly available SAR processing software
  • OpeSARToolkit - High-level functionality for the inventory, download and pre-processing of Sentinel-1 data in the python language.
  • infrastructure - Mapping and monitoring of infrastructure in desert regions with Sentinel-1
  • OST_Notebook - The notebooks within this repository provide getting started tutorials for the use of the Open SAR Toolkit, found here in the ESA-philab github channel.
  • S1_ARD - repository for testing analysis-readiness of Sentinel-1 RTC backscatter Python
  • sea_ice_drift - Sea ice drift from Sentinel-1 SAR imagery using open source feature tracking Python
  • s1prepro - Automated pre-processing of Sentinel 1 (satellite radar imagery) Python
  • Spacenet6 - SAR buildings - The winning solutions for the SpaceNet 6 Challenge Python
  • sentinel1-opds - sentinel1-opds ingestion Python
  • rice_sentinel1 - classify rice from sentinel 1 data Python
  • sentineldenoised - Thermal noise subtraction, scalloping correction, angular correction Python
  • sentinel1-Biodiversity - Code, example dataset, and instructions of Sentinel-1 data pre-processing and pixel-based summary statistics used in "Radar vision for mapping forest biodiversity from space" Python
  • Step by step: Radar-based flood mapping with Python and github link - This repository contains a Jupyter Notebook for automatic flood extent mapping using space-based information. Python
  • STAC Sentinel1 - stactools package for working with sentinel1 data Python
  • sarsen - Algorithms and utilities for Synthetic Aperture Radar (SAR) sensors

LiDAR

  • ICESAT extraction script - Python script to convert from ICESat-2 ATL08 HDF data to shapefile. Usage: 'python icesat2_shp.py
  • ICESAT tools - Tools and code for Icesat-2 data analysis (Python)
  • usgs-lidar - AWS Entwine Point Tiles USGS LiDAR Public Dataset GitHub repo
  • Lidar - Terrain and hydrological analysis based on LiDAR-derived digital elevation models (DEM)
  • IcePyx - Python tools for obtaining and working with ICESat-2 data

GEDI

  • pyGEDI - pyGEDI is a Python Package for NASA's Global Ecosystem Dynamics Investigation (GEDI) mission, data extraction, analysis, processing and visualization.
  • GEDI extraction script - Python script to take GEDI level 2 data and convert variables to a geospatial vector format
  • rGEDI - rGEDI: An R Package for NASA's Global Ecosystem Dynamics Investigation (GEDI) Data Visualization and Processing.
  • pysl4land - Python tools to process spaceborne lidar (GEDI and ICESAT2) for land (pySL4Land) applications
  • gedi - Python tutorial to process and handle LiDAR GIDE datasets
  • sprnca_gedi - WIP to map Foliage Height Diversity along the San Pedro Riparian Corridor with NASA's GEDI Lidar Python
  • GEDI_Yucatan - Supplementary material for the study: Space Lidar for Archaeology? Reanalyzing GEDI Data for Detection of Ancient Maya Buildings R
  • q_research - For processing of ICESat GLAS, GEDI and ICESat-2 LiDAR data, to derive q parameter for canopy height to density relationship Python

InSAR

  • ISCE - InSAR Scientific Computing Environment version 3 alpha
  • LiCSBAS - LiCSBAS package to carry out InSAR time series analysis using LiCSAR products
  • MintPy - Miami InSAR time-series software in Python
  • Pyrocko - Can be utilized flexibly for a variety of geophysical tasks, like seismological data processing and analysis, modelling of InSAR, GPS data and dynamic waveforms, or for seismic source characterization.
  • InSARFlow - Parallel InSAR processing for Time-series analysis
  • PyRate - A Python tool for estimating velocity and time-series from Interferometric Synthetic Aperture Radar (InSAR) data.
  • ARIRA-tools - Tools for exploiting ARIA standard products Python
  • ISCE_utils - Small utility scripts for working with InSAR Scientific Computing Environment (ISCE) products Python
  • ROI_PAC-Sentinel1 - InSAR processing of Sentinel-1 using ROI_PAC
  • insar_scripts - Useful scripts for working with roipac data Python
  • pygmtsar - Collection of Python scripts for InSAR processing with GMTSAR
  • isce2 - InSAR Scientific Computing Environment version 2 Python
  • snap2stamps - Using SNAP as InSAR processor for StaMPS

Landuse

  • demeter - A land use land cover disaggregation and change detection model Python

Visualisation

Regular blogs of significant interest or posts of interest

EO code Competitions

ARD links

  • S1_S2_ARD_code_list - A curated list supporting the use of Sentinel-1 and Sentinel-2 analysis-ready data (ARD) via application programming interface (API)

Useful EO code based twitter accounts

  • pyGEDI - pyGEDI is a Python Package for NASA's Global Ecosystem Dynamics Investigation (GEDI) mission, data extraction, analysis, processing and visualization.

Great Github accounts

Please do explore these accounts, there are some absolutely brilliant projects on these accounts. This was previously a section containing examples, but these are better grouped into the other headings and repitition of links removed. However I feel its very important to highlight individuals wherever possible, ordered by github account name.

| Chis Holden github | Christoph Rieke git hub | Fernerkundung | gena github | jgomezdans github - blog | Johntruckhenbrodt github | Marcus Netler on github | Oliverhagolle github | PerryGeo | giswqs - Qiusheng Wu github | rhammell | Remote pixel github | robintw | Evan Roualt github | samapriya github | shakasom github | yannforget github |

EO Geospatial companies or orgs making big contributions

Github accounts only with examples of work. This section used to contain examples of work, these have been now regrouped into other sections to make them easier to find.

| development seed | mapbox | Planet Labs, now just Planet | Digital Globe - now Maxar | Azavea | Radiant Earth foundation | Sentinel Hub | PyTroll | CosmiQ | Theia software and tools | sparkgeo | Geoscience Australia | Dymaxion Labs | Satellogic | senbox-org | Nasa-gibs | mundialis | ESA_PhiLab | Element 84

Interesting Non EO parts Python

This bit could potentially become the most valuable resource. Lets not ignore other sectors/industries/data science, instead lets embrace it and learn from all that other amazing stuff! This my prelude to saying we are earth data scientists

Interesting Non EO parts other languages

This section is aimed more a data science/programming resources that 'might' be applicable to Earth Observation, that are not Python

Data

I don't really want to add many data resources to this list as it creeps out of scope but this part contains some good data links [not necessarily EO]

  • Environmental_Intelligence - Data for Environmental Intelligence: A mega list of Earth System Datasets covering earth observations, climate, water, forests, biodiversity, ecology, protected areas, natural hazards, marine and the tracking of UN's Sustainable Development Goals
  • gibs - This is EO
  • awesome-gee-community-datasets - Community Datasets added by users and made available for use at large

A footnote on awesome

There are many awesome lists relating to 'Geo'. I use that term as widely as possible. This list is not meant to replace these lists. Earth Observation is still way behind the GIS world in terms of audience, reach, number of users etc. Things are changing though, by bringing these links together I hope you can see that there has been so much progress in the last 5 years. I do hope these links are helpful espcially to those who are new to Earth Observation, but also to people like me who with several years of experience think they may have seen it all - we haven't and there is still so much to learn. Earth Observation is not just an academic 'thing' or a basemap anymore, it forms the basis for a growing and diverse business environment. Lets embrace this.

Finally, I wanted to acknowledge a couple of awesome Earth Observation lists that you may list to check out:

原文：https://github.com/acgeospatial/awesome-earthobservation-code

标签

↑