BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Europe/Stockholm
X-LIC-LOCATION:Europe/Stockholm
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20220812T074334Z
LOCATION:Foyer 2nd Floor
DTSTART;TZID=Europe/Stockholm:20220628T090000
DTEND;TZID=Europe/Stockholm:20220628T110000
UID:submissions.pasc-conference.org_PASC22_sess181_pos146@linklings.com
SUMMARY:P28 - Distributed Training of Deep Neural Networks
DESCRIPTION:Poster\n\nP28 - Distributed Training of Deep Neural Networks\n
 \nKopanicakova, Cruz, Kothari, Krause\n\nDeep networks (DNNs) are nowadays
  used in a wide range of application areas and scientific fields. Since th
 e representation capacity of DNNs is tightly coupled to their width and de
 pth, networks have grown considerably over the last year. As this growing 
 trend is expected to continue, the development of novel, highly-scalable t
 raining algorithms becomes an important task. <br /> <br /> In this work, 
 we propose novel distributed-training strategies for large-scale DNNs. The
  developed training algorithms are based on multilevel and domain decompos
 ition sub-space correction techniques, well-known from numerical computing
 . A hierarchy of suitable sub-spaces, related to different levels or subdo
 mains, is constructed by exploiting the underlying structure of the loss f
 unction and the network architecture. For the implementation, we leverage 
 the PyTorch framework and take advantage of CUDA and NCCL technologies. <b
 r /> <br /> The convergence properties and scaling behavior of the propose
 d training methods will be demonstrated using several state-of-the-art ben
 chmark problems. Moreover, a comparison with the widely-used stochastic gr
 adient optimizer will be presented, showing a significant reduction in the
  number of iterations and the execution time.
END:VEVENT
END:VCALENDAR