Application of the kernel density estimation method for anomaly detection in bivalves activity data

by

A.N. Grekov, E.V. Vyshkvarkova, V.V. Trusevich

E-mail: i@angrekov.ru

Institute of Natural and Technical Systems,

RF, Sevastopol, Lenin St., 28

DOI: 10.33075/2220-5861-2024-4-135-144

UDC 004.852

EDN: https://elibrary.ru/wdzskx

Abstract:

The article discusses the use of the kernel density estimation (KDE) method to anomalies detection in the biological time series of bivalve mollusks activity. The data obtained from the automated biomonitoring system of aquatic environment in the period from February to April 2017 were analyzed with the aim of detecting deviations in the behavior of mollusks associated with a change in environmental conditions, such as water pollution and temperature fluctuations. The KDE allowed to build a smooth approximation of the probabilistic distribution of time series, which ensured the exact identification of rare events characteristic of anomalies. The method was tested on monitoring of mussels Unio Pictorum (Linnaeus, 1758) namely the size of the valve opening and showed high accuracy in modeling normal biological rhythms and identifying anomalies. The results of the study demonstrate that the KDE is an effective and flexible tool for the analysis of complex time series and a can be successfully used for environmental monitoring and predicting negative changes in the environment. The detection time of one of the anomalies in the bivalve activity data using the KDE method turned out to be significantly better compared to previously applied machine learning algorithms.

Keywords: anomaly detection, bivalve mollusks, kernel density estimation, machine learning, biomonitoring.

Full text in PDF(RUS)

REFERENCES

  1. Trusevich V.V., Kuz’min K.A., Mishurov V.Zh., Zhuravsky V.Yu., and Vyshkvarkova E.V. Features of Behavioral Responses of the Mediterranean Mussel in Its Natural Habitat of the Black Sea. Inland Water Biology, 2021, Vol. 14, No. 1, pp. 10–19. DOI: 10.1134/S1995082921010132
  2. Rosenblatt M. Remarks on Some Nonparametric Estimates of a Density Function. The Annals of Mathematical Statistics, 1956, Vol. 27 (3), pp. 832–837. DOI: 10.1214/aoms/1177728190
  3. Sow S., de Vlas S.J., Stelma F.F., Vereecken K., Gryseels B., and Polman K. The contribution of water contact behavior to the high Schistosoma mansoni infection rates observed in the Senegal River Basin. BMC Infectious Diseases, 2011, Vol. 11, pp. 198.
  4. Lang C.I., Sun F.K., Lawler B., Dillon J., Al Dujaili A., Ruth J., and Boning D.S. One class process anomaly detection using kernel density estimation methods. IEEE Transactions on Semiconductor Manufacturing, 2022, Vol. 35 (3), pp. 457–469. DOI:10.1109/TSM.2022.3181468
  5. Nachman B. and Shih D. Anomaly Detection with Density Estimation. Phys. Rev. D, 2020, Vol. 101, Iss. 7, pp. 075042. DOI:https://doi.org/10.1103/PhysRevD.101.075042
  6. Lindstrom M.R., Jung H., and Larocque D. Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection. Entropy, 2020, Vol. 22 (12), pp. 1363. https://doi.org/10.3390/e22121363
  7. Harvey A. and Oryshchenko V. Kernel density estimation for time series data. International Journal of Forecasting, 2012, Vol. 28, Iss. 1, pp. 3–14. DOI: 10.1016/j.ijforecast.2011.02.016
  8. Trusevich V.V., Kuz’min K.A., and Mishurov V.Zh. Monitoring vodnoy sredy s ispol’zovaniyem presnovodnykh dvustvorchatykh mollyuskov (Monitoring of the aquatic environment using freshwater bivalves). Sistemy kontrolya okruzhayushchey sredy, 2017, No. 7 (27), pp. 83–93.
  9. Grekov A.N., Seleznev I.A., Ivakin YA.A., Grekov N.A., Vyshkvarkova E.V., and Trusevich V.V. Biologicheskie sistemy rannego obnaruzhenija dlja jekologicheskogo monitoringa vodnoj sredy (Biological early warning systems for environmental monitoring of the aquatic environment). Sevastopol, INTS, 2023, 131 p.
  10. Silverman B.W. Density Estimation for Statistics and Data Analysis. Published in Monographs on Statistics and Applied Probability, London: Chapman and Hall, 1986.
  11. Scott D.W. Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons, 2015. ISBN: 9780471697558. DOI:10.1002/9781118575574
  12. McKinney W. Data structures for statistical computing in Python. SciPy, 2010, Vol. 445 (1), pp. 51–56.
  13. Harris C.R., Millman K.J., van der Walt S.J., Gommers R., Virtanen P., Cournapeau D., Wieser E., Taylor J., Berg S., Smith N.J., Kern R., Picus M., Hoyer S., van Kerkwijk M.H., Brett M., Haldane A., Fernández del Río J., Wiebe M., Peterson P., Gérard-Marchant P., Sheppard K., Reddy T., Weckesser W., Abbasi H., Gohlke C., and Oliphant T.E. Array programming with NumPy. Nature, 2020, Vol. 585, pp. 357–362. DOI: 10.1038/s41586-020-2649-2
  14. Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., and Duchesnay E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011, Vol. 12, pp. 2825–2830.
  15. Grekov A.N., Vyshkvarkova E.V., and Mavrin A.S. Forecasting and Anomaly Detection in BEWS: Comparative Study of Theta, Croston, and Prophet Algorithms. Forecasting, 2024, Vol. 6 (2), pp. 343–356. https://doi.org/10.3390/forecast6020019
  16. Grekov A.N., Vyshkvarkova E.V., and Mavrin A.S. Algoritm obnaruzhenija anomalij s pomoshh’ju modeli SARIMA dlja programmnogo obespechenija avtomatizirovannogo kompleksa biomonitoringa vodnoj sredy (Аnomaly detection algorithm using the SARIMA model for the software of an automated complex for the aquatic environment biomonitoring). Iskusstvennyj intellekt i prinjatie reshenij, 2024, No. 1, pp. 52–67. DOI: 10.14357/20718594240105

Loading