09/05/2020
👉Interior Point (IP) methods: IP methods
cast the SVM learning task as a quadratic optimization problem subject to linear constraints.
The constraints are replaced with a barrier function. The result is a sequence of unconstrained problems which can be optimized very efficiently using Newton or Quasi-Newton
methods.
👉Decomposition methods: To overcome the quadratic memory requirement of IP methods,
decomposition methods such as SMO and SVM-Light tackle the dual representation of the SVM optimization problem, and employ an active set of constraints thus workingon a subset of dual variables. In the extreme case, called row-action methods, the activeset consists of a single constraint.
👉Primal optimization: Tackling the
primal objective directly was studied, for example, by Chapelle , who considered using smooth loss functions instead of the hinge loss, in which case the optimization problem
becomes a smooth unconstrained optimization problem. Chapelle then suggested using various optimization approaches such as conjugate gradient descent and Newton’s method. We
take a similar approach here, however we cope with the non-differentiability of the hingeloss directly by using sub-gradients instead of gradients.
Now in next post we'll look into codes and kernels.
If you like the content please consider following the page and sharing in your circle ❤️
Just follow the page and keep brushing up your concepts. ;)
For pegasos, you can read this research paper :
https://www.google.com/url?sa=t&source=web&rct=j&url=https://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf&ved=2ahUKEwjny5q586XpAhUr7HMBHTygCO8QFjACegQIAhAB&usg=AOvVaw0NEf9uKZVjxKtrCE7y5KfR
Follow 🤓
@ Mumbai - City of Dreams
![[Share this post on your stories]https://www.datacamp.com/freeweekFree all week! Unlimited access to DataCamp Everyone s...](https://img3.schoolandcollegelistings.com/422/085/140835294220852.jpg)
![[Zoom-In to read] ** OFF TOPIC**Sharing the information about **ar This post shares what the problems look like, ...](https://img5.schoolandcollegelistings.com/431/235/139920254312356.jpg)
