Data clustering in anomalies detection tasks based on orthogonal filters

by

A.V. Skatkov1, Y.E. Shishkin1,2

1 Federal State Educational Institution of Higher Education «Sevastopol State University»,

Russian Federation, Sevastopol, Universitetskaya St., 33

2 Institute of Natural and Technical Systems, Russian Federation, Sevastopol, Lenin St., 28

E–mail: yourockpro@gmail.com

DOI: 10.33075/2220-5861-2018-1-36-43

UDC 004.942:004.75

Abstract:

   An approach to solve the problem of anomalies operative detection in monitoring data using adaptive digital filtering based on an orthogonal filter using FFT decomposition is proposed. A mathematical simulation and a real-time clustering procedure was performed. The work of the proposed approach is exemplified by the sampling Sevastopol water area sampling data using a hydrological SVP probe.

Keywords: monitoring, mathematical modeling, Big Data, digital filtering, discrete Fourier transform, detection of anomalies, clustering, critical systems, data mining.

Full text in PDF (RUS)

LIST OF REFERENCES

  1. Data and process analysis: textbook. manual / A. A. Barseghyan, M. S. Kupriyanov, I. I. Kholod [et al.]. 3rd ed., reprint. St. Petersburg: BHV-Petersburg, 2009. 512 p.
  2. Zubrienko G. A., Laponina O. R. Methods for optimizing data sampling for determining abnormal traffic / International Journal of Open Information Technologies. 2016. Vol. 4. No. 10. P. 1-8.
  3. Skatkov A.V., Shishkin Y. Y. The anomaly detection Model in the fields of observation with the use of parametric monitoring systems // Monitoring systems of environment. Sevastopol: INTS. 2017. Issue 10 (30). P. 48-53.
  4. Grekov A. N., Grekov N. A., Shishkin Yu. E. Research of characteristics of the sound velocity Profiler and correction of measurement results / /Monitoring systems of environment. Sevastopol: INTS. 2017. Issue 10 (30). P. 24-30.
  5. Shishkin Yu. E. Certificate No. 2017664038 Russian Federation. Module for reducing the redundancy of monitoring data “MOSIDAM”: certificate of state registration of a computer program / applicant and copyright holder Yu. E. Shishkin; no. 2017660984; application 27.10.2017, publ. 14.12.2017; bul. no. 12. 1 p.
  6. Shishkin, Y. E. Study of the capabilities of the detection system of borrowing in the methodology for Big Data // Fundamental basis of innovative development of science and education: monograph / under the editorship of G. Yu. Gulyaev. Penza: Science and Education, 2017. P. 55-73.
  7. Wentzel E. S., Ovcharov L. A. Probability Theory and its engineering applications. Moscow: Higher school, 2007. P. 491.
  8. Boev V. D. Conceptual design of systems in Anylogic 7 and GPSS World. M.: NO and Intuit, 2016. P. 556.ISBN: 978-5-9556-0161-8.
  9. Shishkin Y.E. Big Data visualization in decision making // Science in Progress: тез. Всерос. науч.-практ. конф. магистрантов и аспирантов. Новосибирск 20 октября 2016 г. Новосибирск: НГТУ, 2016. C. 203–205.
  10. The International Thermodynamic Equation of Seawater 2010 (TEOS-10): Calculation and Use of Thermodynamic Properties / T.J. Mcdougall, Rainer Feistel, F.J. Millero [et al]. Intergovernmental Oceanographic Commission IOC of Unesco, March 2010. 218 pp.
  11. Devyatkov V. V. Methodology and technology of simulation studies of complex systems: current state and prospects of development: monograph. St. Petersburg: University textbook, 2013. P. 448.

Loading