# spring boot 实验3

**Repository Path**: superlgc/spring_boot_experiment_3

## Basic Information

- **Project Name**: spring boot 实验3
- **Description**: No description available
- **Primary Language**: Java
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-05-20
- **Last Updated**: 2020-12-19

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<h1 align="center">东莞理工学院网络空间安全学院</h1>   
<h1 align="center">实验报告模板</h1>    
<table class="table table-borderless">
    <tr>
        <th width=500 align="left">课程名称：企业级开发框架专题</th>
        <th width=50% align="right">学期：2019年秋季</th>
    </tr>
</table>
<table>
  <tr>
    <th width=150>实验名称</th>
    <td width=50% colspan="3">全球新型冠状病毒实时数据统计应用程序的设计与实现
</td>
    <th width=15%>实验序号</th>
    <td width=20%>三</td>
  </tr>
  <tr>
    <th>姓&nbsp;&nbsp;&nbsp;名</th>
    <td>李贵超</td>
    <th>学&nbsp;&nbsp;&nbsp;号</th>
    <td>201741412214</td>
    <th>班&nbsp;&nbsp;&nbsp;级</th>
    <td>17软卓2班</td>
  </tr>
  <tr>
    <th>实验地点</th>
    <td>无</td>
    <th>实验日期</th>
    <td>2020/5/9</td>
    <th>指导老师</th>
    <td>黎志雄</td>
  </tr>
  <tr>
    <th rowspan="2">教师评语</th>
    <td rowspan="2" colspan="3"></td>
    <th>实验成绩</th>
    <td>评阅教师</td>
  </tr>
  <tr>
    <th>百分制</th>
    <td></td>
  </tr>
  <tr>
    <th>同组同学</th>
    <td colspan="5"></td>
  </tr>
</table>   

### 一、实验目的      
 **1、 掌握使用Spring框架自带的RestTemplate工具类爬取网络数据；**    
 **2、 掌握使用Spring框架自带的计划任务功能；**    
 **3、 掌握使用Apache Commons CSV组件解释CSV文件；**    
 **4、 掌握Java 8的Stream API处理集合类型数据；**    
 **5、 了解使用模板引擎或前端框架展示数据。**   
        
### 二、 实验环境      
 **1、 JDK 1.8或更高版本**    
 **2、 Maven 3.6+**    
 **3、 IntelliJ IDEA**    
     
### 三、 实验任务      
 **1. 通过IntelliJ IDEA的Spring Initializr向导创建Spring Boot项目。**       
![输入图片说明](pic/1.png)    
 
 **2. 添加功能模块：spring MVC、lombok、commons-csv等。**       
```xml
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <optional>true</optional>
</dependency>
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-csv</artifactId>
    <version>1.8</version>
</dependency>
```

 **3. 爬取全球冠状病毒实时统计数据。**    
```
地址：https://gitee.com/dgut-sai/COVID-19/blob/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv
```

 **4. 使用Spring框架自带的RestTemplate工具类爬取数据。**          
```java
String uri = "https://gitee.com/dgut-sai/COVID-19/blob/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv";

RequestEntity<Void> requestEntity = RequestEntity.get(URI.create(uri))
    .header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36")
    .header("Cookie", "")
    .build();

ResponseEntity<String> response = new RestTemplate().exchange(requestEntity, String.class);

return response.getBody();
```
 **5. 分析csv文件的数据结构，定义model类。**        
```java
@Data
@NoArgsConstructor
@AllArgsConstructor
public class EpidemicSituation {

    private String provinceOrState;
    private String countryOrRegion;
    private int totalPat;
    private int currAdd;
    private HashMap<String,Integer> dayDetail;
}
```
      
 **6. 使用Apache Commons CSV组件解释CSV文件。**              
```java
Iterable<CSVRecord> records = CSVFormat
    .EXCEL
    .withHeader("Province/State", "Country/Region", "Lat", "Long")
    .parse(new InputStreamReader(new ByteArrayInputStream(data.getBytes()), StandardCharsets.UTF_8));

    for (CSVRecord record : records) {

        if ("Province/State".equals(record.get("Province/State"))) continue;

        EpidemicSituation es = new EpidemicSituation();
        es.setProvinceOrState(record.get("Province/State"));
        es.setCountryOrRegion(record.get("Country/Region"));
        es.setTotalPat(Integer.parseInt(record.get(record.size() - 1)));
        es.setCurrAdd(Integer.parseInt(record.get(record.size() - 1)) - Integer.parseInt(record.get(record.size() - 2)));

        DateRecord dateRecord = new DateRecord(2020, 1, 22);
        HashMap<String, Integer> map = new HashMap<>();
        for (int i = 4; i < record.size(); i++) {
            map.put(dateRecord.toString(), Integer.parseInt(record.get(i)));
            dateRecord.add(1);
        }
        es.setDayDetail(map);
    }
```
自定义DateRecord类处理时间。     

```java
public class DateRecord {

    private int year;
    private int month;
    private int day;
    private final List<Integer> leapDys = Arrays.asList(31,29,31,30,31,30,31,31,30,31,30,31);
    private final List<Integer> ordinaryDys = Arrays.asList(31,29,31,30,31,30,31,31,30,31,30,31);

    public DateRecord(int year,int month,int day) {
        this.year = year;
        this.month = month;
        this.day = day;
    }

    public Boolean isLeapYear() {
        return this.year%4==0||this.year%100==0&&this.year%400!=0;
    }

    public void addOneDay() {

        List<Integer> days = new ArrayList<>(isLeapYear()?leapDys:ordinaryDys);

        this.day += 1;
        if (this.day > days.get(this.month - 1)) {
            this.month += 1;
            this.day = 1;
        }
        if (this.month > 12) {
            this.year += 1;
            this.month = 1;
        }
    }

    public void add(int days) {
        while (days>0){
            this.addOneDay();
            days -= 1;
        }
    }

    public String toString() {
        return this.year + "-" + (this.month<10?"0"+this.month:this.month) + "-" + (this.day<10?"0"+this.day:this.day);
    }

```
        
 **7. 使用Spring框架自带的计划任务功能定时更新统计数据。**      
```java
@EnableScheduling
public class AnalysisService {

    @Scheduled(cron = "0 0 1 * * *")
    public void upData() throws IOException {}
}
```
设置任务线程池为20
```properties
spring.task.scheduling.pool.size=20
```
  
 **8. 要确保应用程序启动时，获取一次统计数据。**     
```java
@PostConstruct
@Scheduled(cron = "0 0 1 * * *")
public void upData() throws IOException {}
```

 **9. 单元测试。**     
contraller
```java
@SpringBootTest
@AutoConfigureMockMvc
class DisplayControllerTest {

    @Autowired
    private MockMvc mvc;


    @Test
    void returnData() throws Exception {
        mvc.perform(MockMvcRequestBuilders.get("/data/0/0")
                .accept(MediaType.APPLICATION_JSON))
                .andExpect(status().isOk())
                .andDo(MockMvcResultHandlers.print());
    }

    @Test
    void accumulate() throws Exception {
        mvc.perform(MockMvcRequestBuilders.get("/accumulate")
                .accept(MediaType.APPLICATION_JSON))
                .andExpect(status().isOk())
                .andExpect(jsonPath("$.columns").isArray())
                .andDo(MockMvcResultHandlers.print());
    }

    @Test
    void dayAdd() throws Exception {
        mvc.perform(MockMvcRequestBuilders.get("/dayAdd")
                .accept(MediaType.APPLICATION_JSON))
                .andExpect(status().isOk())
                .andExpect(jsonPath("$.rows").isArray())
                .andDo(MockMvcResultHandlers.print());
    }

    @Test
    void countryTotal() throws Exception {
        mvc.perform(MockMvcRequestBuilders.get("/countryTotal")
                .accept(MediaType.ALL))
                .andExpect(status().isOk())
                .andDo(MockMvcResultHandlers.print());
    }

    @Test
    void total() throws Exception {
        mvc.perform(MockMvcRequestBuilders.get("/total")
                .accept(MediaType.ALL))
                .andExpect(status().isOk())
                .andExpect(jsonPath("$.total").isNumber())
                .andExpect(jsonPath("$.added").isNumber())
                .andExpect(jsonPath("$.lastTime").isString())
                .andDo(MockMvcResultHandlers.print());
    }
}
```
service
```java
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE,classes = {AnalysisService.class})
class AnalysisServiceTest {

    @Autowired
    AnalysisService analysisService;

    @Test
    public void testGetData() {
        Assert.assertNotNull(analysisService.getData());
        Assert.assertEquals(96541,analysisService.getData().length());
    }

    @Test
    public void testTotalData() {
        String data = analysisService.totalData();
        Assert.assertNotNull(data);
    }

}
```

 **10. 定义Cotroller控制器。**    
```java
@Controller
public class DisplayController {

    @Autowired
    private AnalysisService analysis;

    @RequestMapping("/")
    public String index() {
        return "index";
    }

    @ResponseBody
    @RequestMapping("/data/{PORS}/{CORR}")
    public List<EpidemicSituation> ReturnData(@PathVariable("PORS") String PORS,@PathVariable("CORR") String CORR) {
        System.out.println((PORS + " " + CORR));
        if (PORS==null||CORR==null) return new ArrayList<>();
        return analysis
                .getList()
                .stream()
                .filter(e -> PORS.equals("0") || e.getProvinceOrState().contains(PORS))
                .filter(e -> CORR.equals("0") || e.getCountryOrRegion().contains(CORR))
                .collect(Collectors.toList());
    }

    @ResponseBody
    @RequestMapping("/accumulate")
    public String accumulate(){
        return analysis.accumulateJson();
    }

    @ResponseBody
    @RequestMapping("/dayAdd")
    public String dayAdd(){
        return analysis.dayAddData();
    }

    @ResponseBody
    @RequestMapping("/countryTotal")
    public String countryTotal(){
        return analysis.countryOrRegionTotalNumber();
    }

    @ResponseBody
    @RequestMapping("/total")
    public String total(){
        return analysis.totalData();
    }
}
```
使用Stream API对数据进行处理   
 ```java
public Stream<Map.Entry<String, Integer>> accumulateData() {

return list.stream()
    .map(EpidemicSituation::getDayDetail)
    .reduce((m1, m2) -> {
        HashMap<String, Integer> temp = new HashMap<>();
        for (Map.Entry<String, Integer> entry : m1.entrySet()) {
            if (m2.containsKey(entry.getKey())) {
                temp.put(entry.getKey(), entry.getValue() + m2.get(entry.getKey()));
            }
        }
        return temp;
    })
    .map(HashMap::entrySet)
    .stream()
    .flatMap(Collection::stream)
    .sorted(Map.Entry.comparingByKey());
}
```

 **9. 定义前端数据展示页面。**   
使用技术：Vue，axios，element
功能和效果如下图所示：
![输入图片说明](pic/2.png)  
### 四、实验总结       
通过这次实验，在实操的过程中我明白了一些课堂上还没有完全理解的问题，收获满满，再次感谢老师！