Change log

3.6.2021 (beta v0)

Breaking changes

  • /document endpoint
    • Deleting document or notification returned status code 204 before. Now they're returning 200 with a body where there is a number of effected documents
    • Before some of the endpoints accepted additional properties. This is no longer the case.
    • Some of the error responses have changed. More about the format can be found on this page. Not all of the error messages have been converted so there will be a mixture of old and new.

New features

  • /warehouse endpoint - see swagger in api.laji.fi for more details
    • New field gathering.conversions.eurefCenterPoint.lat/lon for ETRS-TM35FIN centerpoint
    • New field unit.abundanceUnit (INDIVIDUAL_COUNT, PAIRCOUNT, NESTS, ..., FLOWERS, SPOTS, TRUNKS, ... - enum descriptions: https://schema.laji.fi/alt/MY.abundanceUnitEnum) - This field is only interpreted from LajiGIS data at the moment. Use can be extended in the future. The field contains information about what is counted in the unit.abundanceString field.
    • New fields unit.identifications and unit.types - these contain array of IdentificationEvent and TypeSpecimen objects
    • New field unit.plantStatusCode can have one value of https://schema.laji.fi/alt/MY.plantStatusCodeEnum. This field has very little data so far, will have more in the future once Kastikka plant database records have been reprocessed.
    • New filter plantStatusCode - takes in one or more values of MY.plantStatusCodeEnum.

21.4.2021 (beta v0)

New features

Taxon autocomplete: Two new filters includedNameTypes, includedLanguages that can be used to limit names to only finnish vernacular names, for example (includedNameTypes=MX.vernacularName&includedLanguages=fi)

All endpoints: Multilang elements can now return any language codes (previously just fi, en, sv); there is no list of all the possibilities. Mainly the new languages at this point are ru (russian) and (se) sami.

Taxa endpoint has new features that were needed for Regional Threath Assesment result pages. You can check the queries the service makes if interested in these features.

Warehouse endpoint:

  • New filter: collectionAndRecordQuality - can be used to filter occurrences based on combinations of collection quality raiting and individual occurrence quality
  • latestRedListStatus, taxonomicOrder have been added to /list selectable fields
  • USER_HIDDEN secureReason has been split into more detailed reasons: USER_PERSON_NAMES_HIDDEN, USER_HIDDEN_LOCATION, USER_HIDDEN_TIME
  • New field: taxon.sensitive boolean to all places with references to taxon object - tells if the taxon linked to the occurrence(etc) is on the sensitive species list

30.11.2020 (beta v0)

Breaking changes

FinBIF form model was redesigned. The model wasn't documented before, so all the changes won't be listed here, but the most important changes were:

  • features was removed, and the features were merged into options
  • Lot of properties from the root were moved under options

The new model is documented in the schema (https://schema.laji.fi/class/MHL.form).

23.4.2020 (beta v0)

New features

Authentication: Prototype for mobile app auth scheme flow has been implemented. See documentation.

14.4.2020 (beta v0)

Breaking changes

FinBIF quality control system has been completely redesigned. There are numerous changes to annotations model and a new endpoint /annotations/tags. From the new endpoint you can list all the tags that can be used within the new annotation model.

For the /warehouse endpoint, there are following breaking changes:

  • The following concepts have been depracated. They no longer contain meaningful values and in the future can no longer be used as /list selected or sortBy values (will cause error if used):
    • document.quality.reliabilityOfCollection (numeric value 1-5) - replaced by document.linkings.collectionQuality (enum: PROFESSIONAL, HOBBYIST, AMATEUR)
    • unit.quality.reliable (boolean) - replaced by unit.interpretations.reliability (enum: RELIABLE, UNDEFINED, UNRELIABLE)
    • unit.quality.taxon.reliability (enum: RELIABLE, LIKELY, NEUTRAL, SUSPICIOUS, UNRELIABLE) - replaced by unit.interpretations.recordQuality (enum: EXPERT_VERIFIED, COMMUNITY_VERIFIED, NEUTRAL, UNCERTAIN, ERRONEUS)
    • unit.quality.taxon.source (was used to tell the source of taxon.reliability: USER_MARKING, QUALITY_CONTROL,..) - not replaced
    • unit.interpretations.unidentifiable (boolean; told if unit had been marked as unable to identify) - replaced by unit.interpretations.needsCheck (boolean) and unit.interpretations.needsIdentification (boolean) and quality tag CHECKED_CANNOT_VERIFY (MMAN.18)
    • annotations.annotationClass - replaced by different types of tags
    • annotations.invasiveControlEffectiveness - replaced by tags
    • annotations.opinion - replaced by annotation.identification object
    • annotations.type - replaced by different types of tags
  • The following filters are no longer supported
    • taxonReliability - replaced by recordQuality filter; taxonReliability falls back to use new reliability filter but this support will be removed in later releases and will cause error
    • reliable - replaced by reliability; this filter does not have any effect and will cause error if used after future releases
    • reliabilityOfCollection - replaced by collectionQuality filter; this filter does not have any effect anymore and will cause error if used after future releases
    • annotationType (used in /annotation endpoint to filter different types of annotations) has entirely new value range - old values have no effect and will cause error in the future if used
  • Erroneus (unreliable) occurrences are no longer shown by default. They can be included using recordQuality filter or by setting issues filter to include occurrence with issues.
  • Occurrences have a wild (WILD, UNKNOWN, NON_WILD) status and a filter. NON_WILD occurrences are not shown by default. They can be included using wild filter. (This is an older change that has been in production for some time, but has been left undocumented).

New features

Audio support: API has /audio endpoint that Vihko forms can use to submit audio files just as /image endpoint can be used to submit observation images. The metadata is the same except for two new fields: wavURL, mp3URL. The /warehouse endpoint has been expanded with audio/media related fields and filters.

Authentication also has two new auth sources: Omariista and iNaturalist. (This has no effect to applications using FinBIF auth service).

Taxonomy: Taxa and autocomplete can be filtered using "taxonSets". These are even more informal taxon groups than "informalGroups"; for example "the species used in waterbird count observation scheme", etc.

Taxa information now contains observationCount, observationCountFinland and observationCountInvasiveFinland. These give a (slowly updating) number of occurrences that we have in FinBIF for taxa (including higher level taxa). It can be used for example to sort species to show most common taxa first.

Taxon names include sami and russian names, but there is no content yet.

Warehouse season and timeOfYear filters have been fixed to work when searching with a range that crosses the year change, for example days 300-50.

We have better query logging which allows us to email our active API users conserning future API changes.

API swagger (OpenAPI spesification) has been improved to include response models for /warehouse endpoint queries.

The /graphql endpoint mentioned in 15.1.2020 update goes to production. It can be used to combine lots of queries into a single query, for example fetching data about several taxa from both /taxa and /warehouse endpoints with a single HTTP request. Graphql allows you to name fields in the response just as you like and to return only the fields that you actually require.

Other

Our services have undergone a security evaluation and several fixes have been made as a result.

15.1.2020 (beta v0)

Changes

Emails from api and apitest now have better information about their source.

New features

GraphQL

First versio of GraphQL endpoint was released. It's still in early development so changes can occur. There is a playground available at https://apitest.laji.fi/v0/graphql.

8.1.2020 (beta v0)

Changes

/Warehouse endpoint names have been changed. Old ones will continue to function at least until we are at v1.

  • /query/list -> /query/unit/list
  • /query/aggregate -> /query/unit/aggregate
  • /query/count -> /query/unit/count
  • /query/statistics -> /query/unit/statistics
  • /query/single -> /query/document

10.10.2019 (beta v0)

Breaking changes

Data warehouse API: taxon objects used to have .taxon.qname field. This has been deprecated since 5.3.2019 and has now been removed.

New features

Data warehouse list query: RANDOM added to orderBy. For example ?orderBy=unit.taxonVerbatim,RANDOM or RANDOM:123 (where 123 is a seed)

Data warehouse has a new concept: Sample. Each unit can have several samples, which are preparations/dna extractions/etc. /query/sample/list endpoint has been added, which has some new filters for samples.

Api.laji.fi swagger documentation has been updated to include Data warehouse models. For example /query/list has a full example response model, so does /query/single.

5.3.2019 (beta v0)

Breaking changes

Response of /warehouse/aggregate?geoJSON=true query has been changed. It now returns valid GeoJSON: Before this change, the root contained paging parameters and a 'featureCollection' key, that contained the valid GeoJSON; GeoJSON has now been moved to the root level. The feature has also been expanded: Previously it was possible to use two aggregate fields (lat and lon); Now it is possible to use any number of aggregate fields (also non-coordinate fields). The fields must include two or four coordinate fields of the same type (lat, lon or latMin/Max, lonMin/Max). Coordinate fields will be used to generate the GeoJSON feature and rest of the aggregate values area added as properties of the geometry.

In warehouse/list, warehouse/aggregate etc endpoints 'taxon.qname' will be replaced by 'taxon.id'. Responses now contain both but taxon.qname is deprecated.

/Taxa endpoint TaxonMedia model has been changed. Check out the changes in api swagger description if you use our taxon images. (Scientific name etc have been moved from root level of image to SimpleTaxon object under 'taxon' key.)

New features

Aggregate queries to /warehouse/aggregate with onlyCount=false now include firstLoadDateMin/Max. They tell about the time span the records that match the filters have been loaded into FinBIF.

The /taxa endpoing has undergone a major reimplementation. It now runs on top of ElasticSearch. Data is syncronized nightly as before. We have tried to keep breaking changes to minimal at this point, but there will be some in the future. The most important new features have been created are:

  • Aggregate queries - /taxa/species endpoints can be given aggregateBy and aggregateOrderyBy parameters. These allow grouping species by any field available in the model. The response will contain both a list response and aggregate response. It is possible to define many aggregates with one query. TODO: document the syntax.
  • Ability to browse different checklist and versions of checklists - For example frozen 2018 FinBIF master checklist or frozen IUCN Red Book Evaluation 2019 checklist.
  • Habitats - Information about habitats (primary habitat and secondary habitats) have been added to taxonomic information. It is also now possible to filter /species by habitats.

There are plenty of other minor additions that mostly concerning IUCN Red List assessment data.

1.10.2018 (beta v0)

Breaking changes

Names of date type filters have been changed

  • loadedLaterThan -> loadedSameOrAfter
  • loadedBefore -> loadedSameOrBefore
  • firstLoadedLaterThan -> firstLoadedSameOrAfter
  • firstLoadedBefore -> firstLoadedSameOrBefore
  • annotatedLaterThan -> annotatedSameOrAfter
  • annotatedBefore -> annotatedSameOrBefore

Target and taxonId filters have been changes so that the previously added new features: exactTarget, originalExactTarget, exactTaxonId, originalExactTaxonId are now handled with two parameters: useIdentificationAnnotations (true/false, default true), includeSubTaxa (true/false, default true)

20.5.2018 (beta v0)

Breaking changes

Behavior of /warehouse aggregateBy=gathering.team has been changed. The old behavior can be achieved by using aggregateBy=gathering.team.memberName. There are now three aggregateable fields that relate with verbatim observer names:

  • gathering.team - aggregate using full team name (separate member names catenated together using ; separator)
  • gathering.team.memberName - aggregate using single team member name
  • gathering.team.memberId - aggregate by id of a single team member name (that can be used as a value of teamMemberId filter)

New features

/warehouse endpoint has new filters:

  • teamMember (case insensitive wildcard search) and teamMemberId to allow searching using observer names.
  • exactTarget, originalExactTarget, exactTaxonId, originalExactTaxonId - search by taxon without including subtaxa. ("original" filters use the original reported taxon linking, the others use the possibly quality-control changed linking).
  • typeOfOccurrenceId, typeOfOccurrenceIdNot - filter by taxa that belong or do not belong to a certain occurrence in Finland type (for example exclude imports or search only imports).
  • Aggregate queries: pessimisticDateRangeHandling - Value of this parameter affects how oldestRecord and newestRecord are calculated regarding observations reported as date span. False (default): oldest=min(date.begin), newest=max(date.end). True: oldest=min(date.end), newest=max(date.begin). Note that true can result in oldest > newest. Api user should use the form oldest = oldest <= newest ? oldest : newest.

23.4.2018 (beta v0)

Warehouse API changes

There are changes to the warehouse API. The following endpoints have been added:

  • /query/gathering/aggregate - Aggregate queries using gathering as base (unit filters can not be used, returns count of matching gatherings)
  • /query/document/aggregate - Aggregate queries using document as base (gathering and unit filters can not be used, returns count of matching documents)
  • /query/statistics - Same as /query/aggregate (base is unit) but results come from PRIVATE/UNSECURED data. Limited to certain safe fields and only to certain allowed collections.
  • /query/gathering/statistics - Same as above but using gathering as base.

Mostly for our internal needs (line-transect result service) there are couple additional features: /query/aggregate has a new parameter: includePairCounts that can be used to include pair count sum/max aggregate functions to the result. /query/gathering/aggregate has a new aggregate function linelengthSum (sum of length of routes of gatherings that have a line geometry).

Previously undocumented feature now parametrized: Previously when using /query/aggregate, null values were not included by default. This is now parameterized (excludeNulls parameter). Default is true (does not include nulls).

/private-query/ endpoints have been removed from the public swagger documentation. The endpoints still exist but are not shown because they added unnecessary repetition to the documentation.

New filter has been added: wgs84CenterPoint

19.3.2018 (beta v0)

Breaking changes

  • /taxa/search and /autocomplete - Taxon search result objects have a nameType field. Values of this field have been changed, for example MX.misappliedCircumscription -> MX.hasMisappliedName
  • The values of excludeNameTypes parameter have changed similarly
  • /taxa/descriptions and /taxa/media - Blacklist parameter has been removed (it was used to disallow content from certain sources). It has been replaced with externalSources parameter. The external sources available are still eol:api (Encyclopedia of Life) and natruforskaren:species (https://dina-web.net/naturalist/). Fetching descriptions and media from external sources is significantly slower. Because by default data from external sources is not fetched, default performance of these endpoints has been increased dramatically.

New features

  • /taxa/species and /taxa/children now have includeMedia parameter. There is also a new hasMediaFilter parameter. These can be used for example to fetch all taxa with images under a certain informalGroup or higher taxon.
  • /taxa/search and /autocomplete - new parameter excludedInformalTaxonGroup can be used to exclude matches from certain informal groups.
  • /taxa/.. - Many of these endpoints now have includeHidden parameter. When for example a species is splitted to many species, FinBIF taxonomy experts change the old species into an aggregate group and create the new splitted species as children of the splitted species. Depending on the case, the experts may not want the aggregate to remain visible while browsing the taxonomy. The aggregate may however be very useful as basis of reporting observations and when old observations are linked to the new taxonomy. We have allowed the experts to hide this kind of taxa. When a taxon is hidden, it is not showed by default in api and if it has children, those children are raised upwards in the tree to "replace" the aggregate. If you want to browse the taxonomy with the hidden groups(etc) visible, use includeHidden=true. Default is false.
  • /news - News can be now filtered by tags.

12.2.2018 (beta v0)

Breaking change

  • Property name change in checklist endpoint bibliographicCitation -> dc:bibliographicCitation
  • Property name change in publication endpoint bibliographicCitation -> dc:bibliographicCitation and URI -> dc:URI

New feature

  • We've added new endpoint (HTMLToPDF) to make pdfs from html. This will accept post request with the html in the body. Response will be the pdf itself. This endpoint is in very early stages and there might be changes in the near future.
  • We've added new feature to coordinate endpoint (/coordinates/location). You can use that to find the location information by sending GeoJSONs geometry object there. Please note that this will only return information if the given coordinates are in Finland.
  • We've added new parameter (level) to logger endpoint. This will allow you to only return status of desired level. You can also ask for multiple levels by using comma separated string as parameters value.

28.8.2017 (beta v0)

Breaking change: GET /annotation ja GET /annotation/:id poistuvat apista. Jatkossa annotaatiot saa suoraan warehouse endpointista dokumentin yhteydessä.

20.11.2017 (beta v0)

Breaking change: Validaattorit /documents endpointissa ovat kiristyneet.

Lisätty /documents/validate endpointti johon pystyy lähettämään havaintoerän tarkistettavaksi. Vastauksena on joko status koodi 200, jos kaikki on kunnossa tai 422 jos jokin kentän arvoista ei läpäissyt validaattoreita.

28.8.2017 (beta v0)

Breaking change: /query/aggregate: "orderBy" -parametri on muutettu toimimaan kuten /list -kyselyn vastaava parametri. Ennen annettiin sarakkeen numero; nyt annetaan sarakkeen nimi. Nyt mahdollista käyttää DESC -määrettä.

Breaking change: YKJ-ruutuihin liittyvien kenttien ja filtereiden nimet ovat muuttuneet ja uusia lisätty. Nämä olivat ennen ykj3, ykj4, ykj3Center, ykj4Center. Ne ovat nyt: ykj100km, ykj100kmCenter, .. 50km, 10k, .. ykj1km, ykj1kmCenter.

Breaking change: Kellonaikaan liittyvät kentät on uudelleennimetty. Tunnit olivat timeBegin/End ja ovat nyt hourBegin, hourEnd. Minuutit on lisätty ja ovat minutesBegin, minutesEnd. Lisätty gathering.displayDateTime jossa yhteenkatenoituna eventDate ja tunnit ja minuutit.

Breaking change (etl-schema tiedonsiirtoformaatti ja /list /single vastausformaatti): Muutettu käyttäjien tunnisteiden linkitystä. Havaintoeriin liitetään nyt kahdenlaisia käyttäjätunnisteita: document.editorUserIds ja gathering.observerUserIds. Ensimmäinen ovat dokumentin tallentajat/muokkaajat. Jälkimmäinen on "omat havainnot" hakua varten ne käyttäjätunnukset, jotka on kirjattu havainnoitsijoiksi. Näiden tietojen perusteella linkitetään käyttäjätunnukset käyttäjien Lajitietokeskus identiteettiin. Linkitykset löytyvät document.linkings.editors ja gathering.linkings.observers.

Lisätty filterit joilla voi hakea editorPersonToken:lla (itse tallentamani/muokkaamani havainnot) tai observerPersonToken:lla (omat havaintoni). Parametrit ovat editorPersonToken ja obseverPersonToken. Suljetun puolen /private-query/ apista voi hakea suoraan editorin/observerin person id:llä, mutta tätä ei sallita julkisessa /query apissa. Nämä parametrit ovat editorId ja observerId. Lisäksi on parametrit editorOrObserverPersonToken ja editorOrObserverPersonId jotka hakevat molemilla (OR kysely).

Breaking change: /aggregate unit.linkings.taxon.administrativeGroup -> administrativeGroups; unit.linkings.taxon.informalTaxonGroup -> informalTaxonGroups

Breaking change: /query filter "coordinatesWithin" on poistettu. Filterille "coordinates" voi nyt antaa viimeisenä arvona suhteen (luku välillä 0.0-1.0) jonka täytyy täyttyä haettavan alueen ja havainnon alueen välillä. 0.0 tai tyhjä = koordinaattien pitää mennä edes hiukan lomittain, 1.0 = havainnon alueen täytyy olla kokonaan haettavan alueen sisällä. Tällä hetkellä tukea ei ole muille luvuille kuin 0 ja 1. Tuki suhteen laskemiselle lisätään myöhemmin. Haettava alue annetaan siis nyt muodossa lat:lon:YKJ:suhde tai latmin:latmax:lonmin:lonmax:TYPE:suhde. Viimeistä parametria ei ole pakko antaa.

Dokumentteihin voi nyt tietovarastoon lähetettäessä merkitä laatumerkintöjä, esim document.quality. Katso tietovaraston kentät -sivulta täydellinen listaus.

Lisätty laatuun liittyviä filtereitä käytettäväksi /query -rajapinnoissa: reliabilityOfCollection (Array(1-5)), reliable (true/false), qualityIssues (NO_ISSUES, BOTH, ONLY_ISSUES) jossa default NO_ISSUES ja taxonReliability (Array(Enum)).

Ensimmäisen avulla voidaan hakea havaintoja kokoelmien laatuluokituksella (esim vain 3,4,5). Reliable on yhdistelmäfilteri joka palauttaa joko kaikki "luotettaviksi" tulkitut tai ei-luotettaviksi tulkitut. Jättämällä filterin tyhjäksi saa molemmat. Default on tyhjä. Luotettavuuden määritelmä on, että havainto on määritelty luotettavaksi joko kokoelman laadun tai käyttäjien tai asiantuntijoiden laatumerkintöjen (annotaatioiden) perusteella. QualityIssues -filterin avulla voidaan joko näyttää vain laatuongelmalliset, ne joissa ei ole laatuongelmia tai molemmat. Oletus on että laatuongelmallisia ei näytetä. TaxonReliability filterin avulla voi rajata havaintoja luotettavuusluokituksen avulla. Arvoja voi antaa useita, esim "luotettavat ja todennäköiset".

Lisätty uusi filter: unidentified (boolean) jolla voi rajata ainoastaan lajileen tunnistamattomat havainnot. Havainto on tunnistamaton jos sen ilmoitettu varmuus (reportedTaxonConfidence) on epävarma, taksonilinkitys ei onnistu tai linkitetty taksoni on lajia ylempi taksoni. Arvolla false haetaan käänteisen määritelmän mukaan, eli lajilleen tunnistetut.

Lisätty unit.linkings.taxon rinnalle unit.linkings.originalTaxon joka pitää sisällään alkuperäisessä havainnossa ilmoitettujen tietojen mukaisen taksonilinkityksen (joka voi poiketa varsinaista taksonilinkityksestä, mikäli havaintoon liittyy käyttäjien tai asiantuntijoiden laatumerkintöjä (annotaatioita) jotka yliajavat alkuperäisen taksonimäärityksen). Lisäksi target ja taxonId filtereiden rinnalle on lisätty originalTarget ja originalTaxonId -filterit.