With the Kerttu app you can listen to and identify bird vocalizations while contributing to a citizen science project. Your first task is to assess whether specific vocalizations presented by Kerttu belong to particular bird species, and your second task is to identify all the birds vocalizing in a given recording. With the information you provide, we will be able to train a computer model to classify bird vocalizations based on the species that made them. At the same time you will be developing your skills in bird sound recognition. The project’s ultimate aim is the automatic recognition of bird species from sound recordings so that anyone can identify birds they have recorded. To use the app, you will need to sign in to FinBIF with your user account.
Kerttu is part of the LIFEPLAN project. LIFEPLAN’s aim is to establish, using comprehensive and standardised sampling, what are the patterns and driving processes behind the current state of the world’s biodiversity. Two important LIFEPLAN project goals are to automatically identify biodiversity and enable public participation in the research. Kerttu is the first step in automated bird sound identification combining machine learning and citizen science. With Kerttu you can play an important role in training a computer model to recognise Finnish bird sounds and help develop methods which may one day recognise all the world’s birds, mammals and amphibians. Machine learning and automated data collection can be used to predict the future of biodiversity. Thank you for participating!
Instructions for using Kerttu
The following text explains how to use Kerttu. There are three sections in the app: Declare your level of expertise, Identify letters and Identify recordings, which can be accessed via the navigation menu on the left side of the page. You can also find detailed instructions by clicking the information buttons found within each section of the app. If you have issues using Kerttu, suggestions for improvement or questions, you can contact us by clicking on the envelope icon at the bottom right hand side of the page. Your feedback is most welcome!
Quick start guide
With these five bullet point instructions, you can already get off to a good start with Kerttu, it is not necessary to go through the whole detailed instructions. Kerttu is intuitive and by clicking on the info buttons on each section you will find the most important tips.
- First, log in to FINBIF with your user ID. Login is required to register the species you are will be asked to identify based on your own expertise.
- Start using Kerttu by answering two questions in the Declare your level of expertise section and selecting the bird species you are able recognize
- Then you can continue to the Identify letters section. In this section, Kerttu asks you to identify hundred candidates if they match the species that appears in the letter. Once you have made identifications for 20 letter, you are free to move on to the next section.
- In the Identify recordings section, Kerttu asks you to identify the species that vocalize in a ten-second audio recording. You are not necessarily expected to identify all the vocalizations, but those according to your own abilities.
- You can see how many recognitions you have made in the Results section. For the research purposes it would be ideal if you spent roughly the same amount of time identifying both letters and recordings.
For the first use of Kerttu, you should schedule at least a quarter of an hour to get started. After that, you can use Kerttu even for very short periods of time. You can stop at any time without losing your identifications. Next time, you simply continue where you left off.
Declare your level of expertise
Begin by evaluating your own expertise from the Declare your level of expertise menu on the left side of the page.
- Assess how easily you can identify Finnish bird species vocalizations and how actively you engage in birdwatching.
- Now select all the species you know you can identify from the list below. The list includes all species with more than 50 observations in the Finnish Biodiversity Information Facility (FinBIF) database. Select all species you are able to identify based on at least one of their vocalization types (e.g., song or call), but note that you may not necessarily be asked to recognize sounds from all the species you have selected. Note however, that you may be asked to identify sounds from species that are unfamiliar. In such a case you can skip attempting to identify these sounds.
Your expertise level and the species you choose will be used to determine which species you will be asked to identify. At the time of writing there are relatively few species included in the Kerttu app, but more will be added as the project matures. Selecting all the species you are able to identify is important background information for the second section of the study in which you will be asked to identify all the bird species vocalizing in a recording. Your anonymized data may also be used for research on sound identification and expertise level.
When you have completed declaring your level of expertise you can proceed to the next section, Identify letters.
In this section you will be asked to decide whether the vocalizing species of a candidate (white box in left spectrogram) is the same species you hear in the letter (white box in right spectrogram). You can play the candidate or letter audio by pressing the button at the bottom left of their respective spectrograms. When playback begins, you will see the recording progress represented by the vertical red line overlayed on the spectrogram. We recommended you use headphones when listening to the recordings. Noise-cancelling headphones perform best and will enable you to detect the faintest sounds, but they are not strictly necessary to participate in the study.
You will be asked to assess candidate sounds that the computer model has matched to a letter with a varying level of correlation. The correlation is a model estimate of the similarity of the candidate to the letter spectrogram and its value is shown above the candidate’s spectrogram. The assessments you make across the range of correlation values will help to calculate probabilistic thresholds required for the identification. Note that this means that the candidates are intentionally not supposed to represent sounds which the computer model has determined belong to the species in the letter.
You can adjust the time window around the candidate by modifying the time buffer which will modify the amount of the sound included before and after the candidate itself. If you want to listen to a longer span of the candidate’s recording, you can also click the view button below the spectrograms. The time buffer is especially useful when listening to short sounds which might be difficult to identify outside of their wider context. By increasing the time buffer you will often hear more vocalizations made by the same individual bird, which can help with identification. Any modification to the time buffer will also apply to the letter recording, but the view button under the text longer section of candidate will only apply to the candidate. By default the time buffer is set to one second.
The focus frequency option allows you to narrow the range of frequencies audible around the focal sound and displayed on the spectrogram. When selected (slider displays yes), the frequency band includes 0.5 kHz above and below the focal sound of the candidate and the letter. When off (slider displays no), a frequency range of 0 to 11 kHz is used. Focus frequency will make frequencies outside the frequency band shown in the spectrogram less audible. If there is a lot of noise or sound at other frequencies than the focal sound, and occurring at the same time, you should keep the focus frequency turned on. Focus frequency will make it easier to hear the focal sounds in the candidate and letter. By default, focus frequency is turned on, but sometimes it can be helpful to turn it off and get a better idea of the candidate in the context of other nearby vocalizing birds. There may be other species vocalizing at the same time as the candidate, but at different frequencies, and it can be easier to distinguish these by using the visual cue of the entire frequency band in the spectrogram. Some species’ vocalizations have considerable frequency variation, so simultaneously seeing and hearing the entire frequency band can help to identify them. Because focus frequency fades the upper and lower frequencies (but does not mute them completely), it may be easier to identify some of the more challenging sounds when focus frequency is turned off, as you will hear the whole soundscape in its wider context.
If you are reasonably sure that the bird species vocalizing in the white box of the candidate matches the letter, answer yes to the question: Does the species vocalizing in the candidate correspond with the letter? If you are reasonably sure that the candidate vocalisation does not belong to the same bird species as the letter, or you do not hear any bird species in the candidate frame, answer no. And if you are unsure, answer I don’t know.
When you have answered the question above, Kerttu will move on to the next candidate automatically. For each letter, you will be presented with 100 candidates. You can see how many candidates you have completed at the top left of the page. If you accidentally answer a question incorrectly you may return and correct your answer by clicking on return to previous candidate. After all 100 candidates have been completed, a new letter will open automatically. You can stop the Kerttu session at any time by logging out or closing your browser. Kerttu remembers the candidate where you last were and the next time you log in, the identification will continue from that candidate.
If the letter vocalisation is unfamiliar or you are just unsure of its candidate identifications in general, you can move on to the next letter by clicking skip letter. Please note that if you do so, all identifications you have made for this letter will be discarded. Therefore, if you have already submitted answers for most of a letter’s candidates it is preferable to simply complete the remaining identifications. Your uncertainty in making identifications will be accounted for by the computer model built using Kerttu’s results.
Identification of candidates will continue until all the letters for the species you selected in the section declare your level of expertise have been completed. Kerttu will report when this milestone has been reached. Do not worry if you are unable to complete your full allotment of species and candidates. Your identifications will still be used and your input will be highly valuable!
You are ready to move on to the next section, identify recordings, once you have completed 20 letters or for all the species selected in the section Declare your expertise. Candidate identifications will be used in generating the recordings in this section. For developing the automated identification, it would be valuable if each user recognized both letters and recordings and spend roughly the same amount of time on both. The number of your own identifications can be tracked in the result section.
In this section, you will be asked to identify all the bird species you can hear vocalizing during a ten-second recording. In addition, a few seconds before and after the focal 10 seconds are shown as darkened. These sections of audio are not played, by default, as their role is to help identify sounds that are clipped by the beginning and end of the focal 10 second window. You can listen to these sections with the crop recording tool by clicking the button and using the mouse cursor to draw a rectangle around the part of spectrogram that you want to view and hear. For example, if the vocalization you are interested in begins at the very end of the focal 10 seconds you can crop towards the end of the focal recording and the part the immediate succeeds it. This will allow you to see and hear the area you have cropped in more detail which can help to the identity of the sound of interest. The crop recording tool can be used to focus on any part of the spectrogram and to playback a particular sound occurring at any time. Cropping dampens the surrounding frequencies of the cropped region in a similar manner to the focus frequency tool of the previous section, identify letters. Cropping helps to highlight and focus on a particular sound. If the recording contains low-frequency background noise or other sounds which obstruct identification you can use the crop recording tool to exclude them from playback, making it easier to identify the more important sounds. Click return to default view to restore the recording window when you have finished examining the cropped region.
After listening carefully to the recording, add all the bird species you can identify by entering them one by one into the text field under insert bird species occurring in the recording. This field uses autocompletion and will suggest birds species names as you type. You can enter the species in English, Finnish, Swedish, by scientific name or six-letter abbreviation. Using this input field you can add any of the bird species that was on the list in the section declare your level of expertise. If the recording contains a bird species that is not in this list, you can add it in the text input field below insert another animal species occurring in the recording but not on the above list. You can also add any other animal species you hear that are not birds using this second input field. The possibilities include mammals and amphibians as the project is also interested in developing automatic identification for these taxa. If you do not hear any birds vocalizing in the recording, select no bird sounds audible in the recording.
If you are reasonably sure about your identification of a bird vocalisation ensure that the button in the column occurs certainly is selected for that particular species. If you are unsure about any of the bird vocalizations in the recording, you can instead select the button under occurs possibly. Another option is to include uncertain identifications in the list under insert another animal species occurring in the recording but not on the above list. In this input field you can add identifications for higher taxonomic groups such as genus or class. For example, if you identify a sound as belonging to a thrush, but are unsure of the exact species, you can input “thrushes” (or genus Turdus) into the second input field. The remaining option is to select recording contains bird vocalizations I cannot identify if there are bird sounds in the recording of species you are completely unfamiliar with, or if the sound is too difficult to identify for another reason.
If the recording has a lot of disturbing background noise (e.g., rain, or low-frequency noise caused by wind) and it clearly interferes with your ability to hear vocalisations, note this by selecting recording is of poor quality (disturbing noise, rain, etc.). This will help the computer model to account for recordings that may not be reliable. You can also indicate if a recording contains human activity (traffic, etc.), such as cars, airplanes or other machines. These sounds may also have biological significance and noting them may later help a computer model to recognize the sounds of different types of human activity. If the recording contains human speech or vocalisations indicate this by selecting recording contains human speech or other vocalizations. Marking this may help to develop automatic human speech recognition.
When you have added all the bird and other animal species you can identify and indicated the other important qualities of the recording noted above, you can move on to the next recording by clicking save and move to next recording at the top right of the page. If you accidentally click this button or realise after you click it that you forgot to include an identification, you can go back and amend your identifications by clicking return to previous. If you need to stop before you are done click the save button. By doing so, the identifications you have made for the current recording are saved and the next time you return to Kerttu you can continue from where you left off.
Thank you very much for your participation!