Mechatronics engineering

Mechatronics engineering

Share

Puplic Channel for engineers
#Channel_Bot
https://t.me/Mechatronebot
#Discussion_channel
https://t.m

21/03/2023

📓 Fundamentals of Electronics: Book 1: Electronic Devices and Circuit Applications pdf📎 https://bit.ly/402zmtg

20/03/2023

Nobody has done more for the history of Convolutional Neural Networks than Yann LeCun! Here are his biggest contributions to the field:

- 1989 - He applies the Back-propagation algorithm to the handwritten zip code recognition learning task: https://github.com/Yorko/deep_learning_python_intro/blob/master/papers/LeCun%20backpropagation%20applied%20to%20handwritten%20zip%20code%20recognition.pdf,
https://proceedings.neurips.cc/paper/1989/file/53c3bce66e43be4f209556518c2fcb54-Paper.pdf
- 1990 - proposes the method Optimal Brain Damage for determining the optimal network architecture by removing the least important weights in the network:https://proceedings.neurips.cc/paper/1989/file/6c9882bbac1c7093bd25041881277658-Paper.pdf
- 1993 - Invents the Siamese Neural Network architecture for Signature Verification:https://proceedings.neurips.cc/paper/1993/file/288cc0ff022877bd3df94bc9360b9c5d-Paper.pdf
- 1995 - He applies Convolutional Neural Network to the speech, and time series learning task: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=e26cc4a1c717653f323715d751c8dea7461aa105
- 1998 - He suggest a new CNN architecture (LeNet - Graph Transformer Networks) for the document recognition learning task:http://lushuangning.oss-cn-beijing.aliyuncs.com/CNN%E5%AD%A6%E4%B9%A0%E7%B3%BB%E5%88%97/Gradient-Based_Learning_Applied_to_Document_Recognition.pdf
- 1998 - Creates the THE MNIST Database of handwritten digits - https://cir.nii.ac.jp/crid/1571417126193283840
- 2002 - Suggests many commonly used tricks to improve NN training:http://yann.lecun.org/exdb/publis/pdf/lecun-98b.pdf
- 2005 - Proposes contrastive learning for face verification:http://www.cs.utoronto.ca/~hinton/csc2535_06/readings/chopra-05.pdf
- 2006 - Suggests a new Dimensionality Reduction technique (DrLIM): https://ieeexplore.ieee.org/document/1640964
- 2009 - Proposes a new multi-stage CNN architecture:https://koray.kavukcuoglu.org/publis/jarrett-iccv-09.pdf
- 2010 - Makes MNIST DATABASE available
- 2013 - Demonstrates the power of Deep Learning for labeling and image segmentation tasks:https://www.cs.cmu.edu/~bhiksha/courses/deeplearning/Fall.2016/pdfs/1028/Scene_Labeling.pdf
- 2013 - Invents a new regularization technique, DropConnecthttp://proceedings.mlr.press/v28/wan13.pdf
- 2014 - He presents an integrated recognition, localization and detection framework using Convolutional Networks:https://arxiv.org/pdf/1312.6229.pdf
- 2014 - Introduced a novel approach for applying deep learning to graph-structured data:https://arxiv.org/pdf/1312.6203.pdf
- 2015 - Suggests character level embedding within CNN for text classification:https://proceedings.neurips.cc/paper/2015/file/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf
- 2017 - Introduces new mathematical tools for defining convolution and pooling operations on graphs:https://arxiv.org/pdf/1611.08097.pdf
By the way, have you checked his Deep Learning course from NYU? https://atcold.github.io/pytorch-Deep-Learning/

----
Find more content like this in my newsletter: https://TheAiEdge.io/

19/03/2023

19/03/2023

It has been quite a journey to arrive at a ChatGPT model! It took some time before we thought about modeling language as a probabilistic generative process. NLP studies the interactions between computers and human language and it is also as old as computers themselves.

Warren Weaver was the first to suggest an algorithmic approach to machine translation (MT) in 1949 and this led to the Georgetown experiment, the first computer application to MT in 1955. In 1957, Chomsky established the first grammar theory. ELIZA (1964) and SHRDLU (1968) can be considered to be the first natural-language understanding computer programs.

The 60s and early 70s marked the era of grammar theories. Transformational-generative Grammar by Chomsky in 1965, Case grammar by Fillmore in 1968 recognized the relationship among the various elements of a sentence. In 1969, Collins proposed the semantic network, a knowledge structure that depicts how concepts are related to one another and illustrates how they interconnect. In 1970, Augmented Transition Networks was a type of graph theoretic structure used in the operational definition of formal languages. In 1972, Schank developed Conceptual Dependency Theory to represent knowledge for natural language input into computers.

During the 70s, the concept of conceptual ontologies became quite fashionable. Conceptual ontologies are similar to knowledge graphs where concepts are linked to each other by how they are associated. You can imagine generating sentences by following concepts paths in ontologies. The famous ones are MARGIE (1975), TaleSpin (1976), QUALM (1977), SAM (1978), PAM (1978), Politics (1979) and Plot Units (1981).

The 80s showed a great period of success for symbolic methods. In 1983, Charniak proposed Passing Markers, a mechanism for resolving ambiguities in language comprehension by indicating the relationship between adjacent words. In 1986, Riesbeck and Martin proposed Uniform Parsing, a new approach to natural language processing that combines parsing and inferencing in a uniform framework for language learning. In 1987, Hirst proposed a new approach to resolving ambiguity: Semantic Interpretation.

The 90s saw the advent of statistical models for NLP. It was the beginning of thinking about language as a probabilistic process. In 1989, Balh proposed a tree-based method to predict the next word in a sentence. IBM presented a series of models for statistical machine translation. In 1990 Chitrao and Grishman demonstrated the potential of statistical parsing techniques for processing messages and Brill et al introduced a method for automatically inducing a part-of-speech tagger by training on a large corpus of text. In 1991, Brown proposed a method for aligning sentences in parallel corpora for machine translation applications.

In 2003, Bengio proposed the first neural language model, a simple feed-forward model. In 2008, Collobert and Weston applied multi-task learning with ConvNet. In 2011, Hinton built a generative text model with Recurrent Neural Networks. In 2013, Mikolov introduced Word2Vec which completely changed the way we approach NLP with NN. In 2014, Sutskever suggested a model for sequence-to-sequence learning. In 2017, Vaswani gave us the Transformer architecture that led to a revolution in model performance. In 2018, Devlin presented BERT that popularized Transformers. And in 2022, we finally got to experience ChatGPT that completely changed the way the public perceived AI.

Want your school to be the top-listed School/college in Sanaa?

Click here to claim your Sponsored Listing.

Location

Website

Address


Sanaa
01