연주의 공부 기록

논문 링크 1: Distilling the Knowledge in a Neural Network: 1503.02531논문 링크 2: Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons: 1811.03233논문 링크 3: Relational Knowledge Distillation: 1904.05068논문 링크 4: LARGE SCALE DISTRIBUTED NEURAL NETWORK TRAINING THROUGH ONLINE DISTILLATION: 1804.03235논문 링크 5: Be Your OwnTeacher: Improve the Performance of Convolutional Neural ..

논문 링크: 1503.02531 OUTTA 논문 리뷰 링크: [2025-1] 박서형 - Distilling the Knowledge in a Neural Network [2025-1] 박서형 - Distilling the Knowledge in a Neural Networkhttps://arxiv.org/abs/1503.02531 Distilling the Knowledge in a Neural NetworkA very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predi..

논문 링크: https://arxiv.org/pdf/1605.07146 OUTTA 논문 리뷰 링크: [2025-1] 조환희 - WideResNet(WRN) [2025-1] 조환희 - WideResNet(WRN)WRN(WideResNet)은 residual netowrk의 넓이를 증가시키고 깊이를 감소시킨 모델이다. 신경망의 넓이를 증가한다는 의미는 filter수를 증가시킨다는 것을 의미한다. 즉, WRN은 residual block을 구성하는 convolublog.outta.ai1. 서론 (Introduction)CNN의 발전 추이초기 CNN(AlexNet, VGG, Inception, ResNet 등)은 성능 향상을 위해 네트워크의 깊이(deep)를 증가시키는 방향으로 발전해 왔다.그러나 모델이 깊..

논문 링크: An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling OUTTA 논문 리뷰 링크:1. Abstract & Introduction배경시퀀스 모델링(sequence modeling) 분야에서, 대부분 RNN(LSTM, GRU 등)이 사실상 디폴트(기본)로 사용되어 왔음.최근 CNN 기반 아키텍처가 오디오 합성, 기계 번역, 언어 모델링 등에서 RNN을 능가한다는 연구 결과가 발표됨.핵심 질문"새로운 시퀀스 태스크를 마주했을 때 RNN vs. CNN 중 어떤 모델을 써야 할까?"RNN이 ‘시퀀스 모델링 = RNN’이라는 공식을 유지할 만큼 여전히 최적일까?연구 목표Convolution..

논문 링크: 1709.01507 OUTTA 논문 리뷰 링크: [2024-2] 박지원- SENet(Squeeze-and-Excitation Networks) [2024-2] 박지원- SENet(Squeeze-and-Excitation Networks)#Squeeze-and-Excitation Networks (2017) Paper ) https://arxiv.org/abs/1709.01507  Squeeze-and-Excitation NetworksThe central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informatblog.o..

논문 링크: https://arxiv.org/abs/1609.02907 Semi-Supervised Classification with Graph Convolutional NetworksWe present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a locarxiv.org OUTTA 논문 리뷰 링크: [2024-2] 이승섭..

NasNet 논문 링크: https://arxiv.org/abs/1707.07012  Learning Transferable Architectures for Scalable Image RecognitionDeveloping neural network image classification models often requires significant architecture engineering. In this paper, we study a method to learn the model architectures directly on the dataset of interest. As this approach is expensive when the datasetarxiv.org MnasNet 논문 링크: htt..

논문 링크1: https://arxiv.org/pdf/1609.04747논문 링크2: https://arxiv.org/pdf/1412.6980논문 링크3: https://arxiv.org/pdf/1711.05101 OUTTA 논문 리뷰 링크: [2024-2] 유경석 - Optimizer의 종류와 특성 [2024-2] 유경석 - Optimizer의 종류와 특성https://arxiv.org/pdf/1609.04747https://arxiv.org/pdf/1412.6980https://arxiv.org/pdf/1711.05101 0. Gradient Descent란?Gradient descent는 model parameter $\theta$에 대한 손실함수 $J(\theta)$를 최소화시키기 위해서, 현재 ..

논문 링크: [PDF] Learning representations by back-propagating errors | Semantic Scholar https://www.semanticscholar.org/paper/Learning-representations-by-back-propagating-errors-Rumelhart-Hinton/052b1d8ce63b07fec3de9dbb583772d860b7c769 www.semanticscholar.org OUTTA 논문 리뷰 링크:[2024-2] 김영중 - Learning representations by back-propagating errors [2024-2] 김영중 - Learning representations by back-propagating ..