Cloud Computing

Cloud Computing
Due to the「COVID-19」 corona-virus disease, this course will adapt on-line course via Microsoft Teams

Schedule and Score (Thursday, 9:10am~12:00am GMT+8 (Taipei), 10 weeks from 2020/11/26 to 2021/1/28)

(AWS ACF Practices)
AWS Cloud Practitioner (AWS ACF) Certification
(0) Login AWS Training & Certification Portal (You have received one invitation email)
Registration Link: 20200520 Academy Cloud Foundations v2 (Asia University)
Prepare for Your AWS Certification Exam
(1) AWS ACF hands-on labs
(2) On-Line video Course
(3) white paper
(4) AWS Cloud Practitioner Practices ((Cost: 20 USD))
(5) AWS Cloud Practitioner Exam ( Cost: 100 USD )
AWS Certified Cloud Practitioner
Sample Questions
(5) Report (with AWS ACF Training and ACF practices Exam) + YouTube (3~5 min)

(Project: Travel Time Analysis of Vehicles in TDCS (AWS EMR)

Presentation :2021/1/28
Report : 2020/1/30 (submit to Moodle with your report and presentation shared in Youtube )
TDCS project with MapReduce programming on Single node Windoop on AWS VMs)

Dataset: (1) 2020/3/2-29 (Four weeks) (2) 2020/3/30-4/5 (One week) (4/2-4/5 : Taiwan Holidays)

Big Data Processing Project: TDCS-06A How to use Web Robot(pdf)

Java Web Robot Example: Web Robot(TDCS_WebURLDownload_jdwang_2017_10_20.zip)

Please select one gantry you favor to observe the variations of 24 hours frequency distribution
Choose one Gantry you favor to observe on Google Map
(1) Is there any significant differences existed for every seven days (one week) when you compare the Dataset (1) and (2)?
(2) Can you have the comparison according to different types of vehicles (31,32, 41, 42, 5)?
(3) How is the computation time in AWS VMs? What is the spec of your VM hardware ?
(4) What is the fee (charge) (AWS Bill) for your computation? How do you think about using AWS EC2 for this middle project?
(You may check with AWS Trusted Advisor to adjust your choices)
(5) Report with the results in(1)(2)(3)(4) and explain with your own words via YouTube (URL shared and embedded within your report)

References for Middle Project

(Gantry Information)（國道計費門架座標及里程牌價表104.09.04版.csv)
(How to import the Gantry locations into GoogleMap?)高速公路計費匝道位置-Google Map 匯入教學

The locations of the Gantries in the National freeway Example: "03F-186.0S"=> GantryID="03F1860S"

TDCS Gantry parsing(24 hours)(pdf)
The frequency distribution of "VehicleType", "GantryID" or "Specific GantryID" within 24 Hours)
(1) Choose one Gantry you favor to observe on Google Map

(Hadoop MapReduce Program: Project for TDCS Gantry parsing (24 hours)
TDCS_GIDSequence_GantryID_VihicleType_Date_Weekday_24Hour_Statistics_jdwang_2018_10_12.zip

(1) (main&mapper){String TargetGantryID = "01F0557N";
(2) modify parameter :　input path
(3) modify parameter :　output path

Testing Data
(One hour)TDCS_M06A_20161127_230000.csv
(one day: 24 Hours)(201701_1-1.7z)
(2018_9_1-7.7z)
Result: part-r-00000_2018_9_1-7＿01F0557N.xlsx
(You can write your own python code to further have these statistics with data visualization)

(15%)(Homework 2: Hadoop Cluster Comparison, Deadline: 2020/5/28)
(0) Windoop Cluster Setup

Windoop_Cluster.html (I627)

(1) AWS EMR Setup

AmazonEMR.html

(2) Run your middle project at Windoop Cluster and AWS EMR, respectively
SpeedUp Experiment （WindoopExecuteTime.xlsx）
SingleNode vs. Multinodes (1,2,4,8)?
small dataset vs. large dataset
many small files (? MB << 128 MB) vs. one packed files (e.g. ?GB)

(3) Comparing Computational Time and Cost via Windoop Cluster or AWS EMR (1 master + 2 workers))
(4) Report (Moodle) + Youtube (3~5 min)

Reference:

AWS Educate Program

AmazonEC2.html

AmazonS3.html

AWS_Training_Certification.html

Windoop_SingleNode.html