Distributed classification learning based on nonlinear vector support machines for switching networks

Yinghui Wang; Peng Lin; Huashu Qin

Kybernetika (2017)

  • Volume: 53, Issue: 4, page 595-611
  • ISSN: 0023-5954

Abstract

top
In this paper, we discuss the distributed design for binary classification based on the nonlinear support vector machine in a time-varying multi-agent network when the training data sets are distributedly located and unavailable to all agents. In particular, the aim is to find a global large margin classifier and then enable each agent to classify any new input data into one of the two labels in the binary classification without sharing its all local data with other agents. We formulate the support vector machine problem into a distributed optimization problem in approximation and employ a distributed algorithm in a time-varying network to solve it. Our algorithm is a stochastic one with the high convergence rate and the low communication cost. With the jointly-connected connectivity condition, we analyze the consensus rate and the convergence rate of the given algorithm. Then some experimental results on various classification training data sets are also provided to illustrate the effectiveness of the given algorithm.

How to cite

top

Wang, Yinghui, Lin, Peng, and Qin, Huashu. "Distributed classification learning based on nonlinear vector support machines for switching networks." Kybernetika 53.4 (2017): 595-611. <http://eudml.org/doc/294851>.

@article{Wang2017,
abstract = {In this paper, we discuss the distributed design for binary classification based on the nonlinear support vector machine in a time-varying multi-agent network when the training data sets are distributedly located and unavailable to all agents. In particular, the aim is to find a global large margin classifier and then enable each agent to classify any new input data into one of the two labels in the binary classification without sharing its all local data with other agents. We formulate the support vector machine problem into a distributed optimization problem in approximation and employ a distributed algorithm in a time-varying network to solve it. Our algorithm is a stochastic one with the high convergence rate and the low communication cost. With the jointly-connected connectivity condition, we analyze the consensus rate and the convergence rate of the given algorithm. Then some experimental results on various classification training data sets are also provided to illustrate the effectiveness of the given algorithm.},
author = {Wang, Yinghui, Lin, Peng, Qin, Huashu},
journal = {Kybernetika},
keywords = {nonlinear support vector machine; multi-agent system; distributed optimization; connectivity},
language = {eng},
number = {4},
pages = {595-611},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Distributed classification learning based on nonlinear vector support machines for switching networks},
url = {http://eudml.org/doc/294851},
volume = {53},
year = {2017},
}

TY - JOUR
AU - Wang, Yinghui
AU - Lin, Peng
AU - Qin, Huashu
TI - Distributed classification learning based on nonlinear vector support machines for switching networks
JO - Kybernetika
PY - 2017
PB - Institute of Information Theory and Automation AS CR
VL - 53
IS - 4
SP - 595
EP - 611
AB - In this paper, we discuss the distributed design for binary classification based on the nonlinear support vector machine in a time-varying multi-agent network when the training data sets are distributedly located and unavailable to all agents. In particular, the aim is to find a global large margin classifier and then enable each agent to classify any new input data into one of the two labels in the binary classification without sharing its all local data with other agents. We formulate the support vector machine problem into a distributed optimization problem in approximation and employ a distributed algorithm in a time-varying network to solve it. Our algorithm is a stochastic one with the high convergence rate and the low communication cost. With the jointly-connected connectivity condition, we analyze the consensus rate and the convergence rate of the given algorithm. Then some experimental results on various classification training data sets are also provided to illustrate the effectiveness of the given algorithm.
LA - eng
KW - nonlinear support vector machine; multi-agent system; distributed optimization; connectivity
UR - http://eudml.org/doc/294851
ER -

References

top
  1. Ali, R., Recht, B., Random features for large-scale kernel machines., In: Advances in Neural Information Processing System, MIT Press, Massachusetts 2008, pp. 1177-1184. 
  2. Bernhard, S., Smola, A. J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond., MIT Press, Massachusetts 2002. 
  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., 10.1561/2200000016, Foundations and Trends in Machine Learning 3 (2011), 1-122. DOI10.1561/2200000016
  4. Chang, C. C., Lin, C. J., 10.1145/1961189.1961199, JACM Trans. Intell. Systems Technol. 2 (2011), 1-27. DOI10.1145/1961189.1961199
  5. Chapelle, O., 10.1162/neco.2007.19.5.1155, Neural Computation 19 (2007), 1155-1178. MR2309267DOI10.1162/neco.2007.19.5.1155
  6. Chapelle, O., Zien, A, Semi-supervised classification by low density separation., In: Proc. International Conference on Artificial Intelligence and Statistics, Barbados 2005. 
  7. Cortes, C., Vapnik, V., 10.1007/bf00994018, Machine Learning 20 (1995), 273-297. DOI10.1007/bf00994018
  8. Drineas, P., Mahoney, M. W., On the Nyström method for approximating a Gram matrix for improved kernel-based learning., J. Machine Learning Research 6 (2005), 2153-2175. MR2249884
  9. Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R., Advances in Knowledge Discovery and Data Mining., AAAI Press, Menlo Park 1996. 
  10. Flouri, K., Beferull-Lozano, B., Tsakalides, P., 10.1109/icdsp.2009.5201180, In: 16th European Signal Processing Conference, Lausanne 2008. DOI10.1109/icdsp.2009.5201180
  11. Forero, P A., Cano, A., Giannakis, G. B., Consensus-based distributed support vector machines., J. Machine Learning Research 11 (2010), 1663-1707. MR2653352
  12. Franc, V., Sonnenburg, S., 10.1145/1390156.1390197, In: Proc. 25th International Conference on Machine Learning, Helsinki 2008. MR2563979DOI10.1145/1390156.1390197
  13. Hu, J., On robust consensus of multi-agent systems with communication delays., Kybernetika 45 (2009), 768-784. Zbl1190.93003MR2599111
  14. Joachims, T., Finley, T., Yu, C. J., 10.1007/s10994-009-5108-8, Machine Learning 77 (2009), 27-59. DOI10.1007/s10994-009-5108-8
  15. Lee, S., Wright, S. J., 10.1007/978-1-4614-5076-4_5, In: Mathematical Methodologies in Pattern Recognition and Machine Learning, Springer, New York 2011, pp. 67-82. DOI10.1007/978-1-4614-5076-4_5
  16. Lou, Y., Hong, Y., Wang, S., 10.1016/j.automatica.2016.02.019, Automatica 69 (2016), 289-297. Zbl1338.93026MR3500113DOI10.1016/j.automatica.2016.02.019
  17. Lu, Y., Roychowdhury, V., Vandenberghe, L., 10.1109/tnn.2007.2000061, IEEE Trans. Neural Networks 19 (2008), 1167-1178. DOI10.1109/tnn.2007.2000061
  18. Kim, W., Park, J., Yoo, J., Kim, H. J., Park, C. G., 10.1109/tsmcb.2012.2226151, IEEE Trans. Cybernetics 43 (2013), 1189-1198. DOI10.1109/tsmcb.2012.2226151
  19. Kim, W., Stanković, M. S., Johansson, K. H., Kim, H.J., 10.1109/TCYB.2014.2377123, IEEE Trans. Neural Cybernetics 45 (2015), 2599–2611. DOI10.1109/TCYB.2014.2377123
  20. Nedic, A., Asuman, O., 10.1109/tac.2008.2009515, IEEE Trans. Automatic Control 54 (2009), 48-61. MR2478070DOI10.1109/tac.2008.2009515
  21. Platt, J. C., Fast training of support vector machines using sequential minimal optimization., In: Advances in Kernel Methods, MIT Press, Massachusetts 1999, pp. 185-208. 
  22. Polyak, B. T., Introduction to Optimization., Springer, New York 1987. MR1099605
  23. Rifkin, R., Klautau, A., In defense of one-vs-all classification., J. Machine Learning Research 5 (2004), 101-141. MR2247975
  24. Scardapane, S., Fierimonte, R., Lorenzo, P. D., Panella, M., Uncini, A., 10.1016/j.neunet.2016.04.007, Neural Networks 80 (2016), 43-52. DOI10.1016/j.neunet.2016.04.007
  25. Shalev-Shwartz, S., Singer, Y., Srebro, N., 10.1145/1273496.1273598, In: Proc. 24th International Conference on Machine Learning, Oregon 2007. DOI10.1145/1273496.1273598
  26. Sra, S., Nowozin, S., Wright, S. J., Optimization for Machine Learning., MIT Press, Massachusetts 2012. 
  27. Wang, X., Chen, Y., 10.14736/kyb-2016-3-0427, Kybernetika 52 (2016), 427-440. MR3532515DOI10.14736/kyb-2016-3-0427
  28. Weston, J., Watkins, C., Support vector machines for multi-class pattern recognition., ESANN 99 (1999), 219-224. 
  29. Yi, P., Hong, Y., 10.1007/s11768-015-5100-8, Control Theory Technol. 13 (2015), 333-347. MR3435158DOI10.1007/s11768-015-5100-8
  30. Yuan, D., Ho, D. W. C., Hong, Y., 10.1137/15m1048896, SIAM J. Control Optim. 54 (2016), 2872-2892. MR3561770DOI10.1137/15m1048896

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.