SelectingSurveySites_Blacklip.Rmd

---
title: "Selecting locations for Timed-swim surveys Post 2020"
author: "Jaime McAllister and Craig Mundy"
date: "`r Sys.Date()`"
output:
  pdf_document:
    fig_height: 8
    fig_width: 8
    includes:
      in_header: header.tex
    toc: yes
  word_document:
    fig_caption: yes
    toc: yes
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(comment = "",     ## nothing goes in front of each line
                      message = FALSE,  ## into the console, not the document
                      warning = FALSE,  ## into the console, not the document
                      fig.align = "center",
                      fig.show = "hold",
                      out.height = "0.8\\textwidth",
                      out.width = "0.8\\textwidth")
options(knitr.kable.NA = '')


suppressPackageStartupMessages({
library(tictoc)
library(sf)
library(nngeo)
library(sp)
library(spatialEco)  
library(lubridate)
library(tidyverse)
library(tmap)
library(tictoc)
library(spdep)
library(openxlsx)  
})

## Set EPSG CRS

GDA2020 <- st_crs(7855)
GDA94 <- st_crs(28355)
WGS84 <- st_crs(4326)

```
# Read in hex layer
Read in grid wide spatial layer, and transform to GDA2020. This hex cell layer contains the majority of fishing activity in the closed blocks.

```{r prepare base layers}
## Read in abalone wide hex layer
ab.hex <- readRDS(paste(sprintf("C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/GIS/SpatialLayers/g1hawide_2020_03_24.rds", Sys.info()[["user"]])))

## Convert to sf
sf.ab.hex <- st_as_sf(ab.hex)

## Transform from GDA94 to GDA2020
sf.ab.hex <- st_transform(sf.ab.hex, GDA2020)

```
# Add quartile to dataframe
THe input variable is the total catch in Kg for each cell from 2012 - 2019. We use the dplyr ntile() function to calculate five quantiles (0-20, 20-40, etc) based on total catch. Given that there is negligible fishing in 2018 and 2019, we could sum catch across 2012-2017 and use an variation if we think the 2018/2019 fishing events are sufficinetly unusual.

```{r add quartile var}
# Filter hex layer to required block and add quartile
ts.hex <- filter(sf.ab.hex, zone == 'E' &
                  blockno %in% c(16, 22, 23, 24, 27)|
                  subblockno == '28A') %>%
  mutate(subblockno = gsub(" ", "", subblockno)) %>%
  within({
    cell.ntile <- ntile(blkgtotal, 5)
  })

# ts.hex <- filter(sf.ab.hex, zone == 'E' &
#                   subblockno %in% c('13E')) %>%
#   mutate(subblockno = gsub(" ", "", subblockno)) %>%
#   within({
#     cell.ntile <- ntile(blkgtotal, 5)
#   })

# # Hex layer and quartile for entire state
# ts.hex <- sf.ab.hex %>%
#   mutate(subblockno = gsub(" ", "", subblockno)) %>%
#   within({
#     cell.ntile <- ntile(blkgtotal, 5)
#   })

outname.ts.hex <- sprintf(paste(sprintf("C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/GIS/SpatialLayers/TimedSwimSites_Hexcell_BL_quantiles_Block_16_28A_2020.gpkg", Sys.info()[["user"]])))

st_write(ts.hex, dsn = outname.ts.hex, layer = "pointqt", driver = "GPKG", append = F)

hex.cell.summary <- ts.hex %>%
  st_set_geometry(NULL) %>%
  group_by(cell.ntile) %>%
  summarise(
    mncatch = mean(blkgtotal, na.rm = T),
    catch = sum(blkgtotal, na.rm = T),
    n = n()
  ) %>%
  print()

# calculate mean cpue for each oid within timed swim survey areas and create data frame for later analysis
df_cpue <- ts.hex %>% 
 mutate(cpue_kg_hr = blkgtotal / (minstotal / 60)) %>%
 st_centroid(.) %>% 
 st_as_sf(coords = c("x", "y")) %>% 
  st_set_crs(st_crs(7855)) %>%
  st_transform(crs = st_crs(4326)) %>%
  sfheaders::sf_to_df(fill = T) %>%
  select(oid, zone, blockno, subblockno, cpue_kg_hr, cell.ntile, x, y) %>% 
  dplyr::rename('latitude' = y,
                'longitude' = x) %>%
 select(-c(latitude, longitude)) %>% 
 distinct()

gis_folder_path <- paste(file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/GIS/SpatialLayers/TimeSwimLayers',
Sys.info()[["user"]]), sep = '')))

write.xlsx(df_cpue, paste(gis_folder_path, paste('/TimedSwimSites_OID_CPUE_Block_16_28A_2020.xlsx', sep = ""), sep = ''), sheetName = "Sheet1", colNames = TRUE, rowNames = TRUE, append = FALSE)

```
# Reference Site Selection from 2020
Randomly select 15 sites from each block to keep as reference sites for 2021 survey and beyond. These sites are selected from the compiled 2020 timed swim raw data and therefore contain a combination of sites originally derived from GPS diver logger data (herein) and historical size-at-maturity (SAM) research sites. In summary, 78 reference sites are derived from the GPS logger data and 12 sites from historical SAM research data. The coordinates are the adjusted position recorded at the time of baseline survey in 2020 and will differ slightly from the original proposed position. Generally the position remained within an OID however some sites may have required further repositioning (e.g. >75 m) and therefore may fall outide the original OID.

NOTE: set.seed wasn't used when generating the selection of reference sites from 2020 to use in 2021. Therefore the original output gpkg file of reference sites has been re-imported to identify those reference sites and stored as an RDS for future site selection. Coding has been deactivated for saving the original selection of reference sites. Also note files were transferred to UTAS Research Dropbox in 2023 when Cloudstor was decommissioned.  

```{r reference site selection 2021}

# Import timed swim compiled raw data from 2020 survey
# time.swim.dat.final <- readRDS('C:/CloudStor/Shared/DiveFisheries/Abalone/FISdata/FIS_TimedSwimSurveys2021/time.swim.dat.final.RDS')

# Set seed
set.seed(123) 

# Randomly select 15 sites from 2020 data
sites.2020.keep <- time.swim.dat.final %>%
 filter(sampyear == 2020,
        !subblockno %in% c('28B', '28C')) %>% 
 group_by(blockno) %>% 
 distinct(site, .keep_all = T) %>% 
 sample_n(15) %>% 
 select(-proposed.geom) %>% 
 st_as_sf()

# save GIS spatial layer
# st_write(sites.2020.keep,
#          dsn = 'C:/CloudStor/Shared/DiveFisheries/Abalone/FISdata/FIS_TimedSwimSurveys2021/FIS_TIMEDSWIMSITES_2020REFERENCE_2021-05-17.gpkg', layer = 'sites.2020.keep', driver = 'GPKG', overwrite = T, delete_dsn = T)

# # convert to data frame
# sites.2020.kept.df <- sites.2020.kept %>%
#  sfheaders::sf_to_df(fill = T)

# save file
# saveRDS(sites.2020.kept.df, 'C:/CloudStor/Shared/DiveFisheries/Abalone/FISdata/FIS_TimedSwimSurveys2021/time.swim.2020.repeat.sites.RDS')

# Import original reference site selection 
# sites.2020.kept.sf <- st_read(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata/FIS_TimedSwimSurveys2021/FIS_TIMEDSWIMSITES_2020REFERENCE_2021-05-17.gpkg', Sys.info()[["user"]])))
#
# # Convert to data frame
# sites.2020.kept.df <- sites.2020.kept.sf %>%
#  sfheaders::sf_to_df(fill = T)

# Save file
# saveRDS(sites.2020.kept.df, paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata/FIS_TimedSwimSurveys2021/time.swim.2020.repeat.sites.RDS', Sys.info()[["user"]])))
#
# saveRDS(sites.2020.kept.df, paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata/FIS_TimedSwimSurveys2021/time.swim.2020.repeat.sites_COPY.RDS', Sys.info()[["user"]])))

```
```{r refsites}
# Import reference sites
time_swim_ref_sites <- readRDS(file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata',
Sys.info()[["user"]])), paste('FIS_TimedSwimSurveys2021'), 'time.swim.2020.repeat.sites.RDS', sep = '')) 

# Denote reference sites with 'ref'
time_swim_ref_sites <- time_swim_ref_sites %>% 
 mutate(ref_site = 'ref')

# Create vector of OIDs for reference sites to be sampled in assessment year 
# Note: 12 of the randomly selected reference sites are historical SAM research sites (i.e. n = 78) and
# have unallocated OIDs
ref_sites <- time_swim_ref_sites

ref_sites_oid <- ref_sites %>% 
        filter(!is.na(oid)) %>% 
        ungroup() %>% 
        pull(oid)

# Replace periods (e.g. '.') in column names
names(time_swim_ref_sites) <- gsub(x = names(time_swim_ref_sites), pattern = '\\.', replacement = '_')

# UPDATE: 2024-10-12 Create dataframe of reference sites to include in each assessment year folder.

# ts_ref_site_dat <- time_swim_ref_sites %>% 
#  select(c(site, site_new, blockno, subblockno, oid, cell_ntile, sam_count, x, y, ref_site)) %>% 
#  dplyr::rename(site_old = 'site',
#                longitude = 'x',
#                latitude = 'y') %>% 
#  st_as_sf(coords = c("longitude", "latitude"), crs = GDA2020) %>% 
#  st_transform(crs = WGS84) %>% 
#  sfheaders::sf_to_df(fill = T) %>% 
#  dplyr::rename('latitude' = y,
#                'longitude' = x) %>% 
#  select(-c(sfg_id, point_id))
# 
# # Export Excel version of reference sites to survey year folder
# data_folder <- file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata', 
#                                             Sys.info()[["user"]])), paste('FIS_TimedSwimSurveys2021', sep = ''))
# 
# ts_ref_site_dat %>% 
#  write.xlsx(file = paste(data_folder, paste('/TimedSwimSites_EastCoast_ReferenceSites_Final.xlsx', sep = ''), sep = ''), sheetName = "Sheet1", col.names = TRUE, row.names = TRUE, append = FALSE)

```

# Post 2020 Site Selection
Timed swim surveys in subsequent years after the collection of baseline data in 2020 involve a semi-repeated measures experimental design where 15 sites from each block sampled in 2020 have been carried over as 'reference' sites with the remaining 45 sites randomly selected from the GPS logger data after excluding all previous GPS logger sites sampled in the previous year (e.g. in 2021 sites surveyed in 2020 (n = 229) were excluded from random selection). 

```{r post 2020 site selection}
## Post 2020 site selection ####

# Specify assessment year
assess_year <- 2024

# Import final timed swim dataframe from previous assessment year
time.swim.dat.final <- readRDS(file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata',
Sys.info()[["user"]])), paste('FIS_TimedSwimSurveys', assess_year - 1, sep = ''), 'time.swim.dat.final.RDS', sep = ''))                                  

outname_point <- file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata',
Sys.info()[["user"]])), paste('FIS_TimedSwimSurveys', assess_year, sep = ''))

# Create vector of all OIDs sampled in previous assessment year to filter from
# current assessment year random site selection
ts_sites_sampled <- time.swim.dat.final %>%
        filter(!subblockno %in% c('28B', '28C'),
               sampyear == assess_year - 1) %>% 
        group_by(blockno) %>% 
        distinct(site, .keep_all = T) %>% 
        filter(!is.na(oid)) %>% 
        ungroup() %>% 
        pull(oid)

# Create vector of blocks
blocknos <- c(16, 22, 23, 24, 27, 28)

# Create empty lists to populate with random hex cells for each block 
ts_site_list <- list()

# Set seed
set.seed(1234)

# Run loops to generate random hex cells for each closed and reference block
for (i in blocknos){
cellqt <- ts.hex %>% 
  filter(!oid %in% ts_sites_sampled &
           cell.ntile > 2 &
           !subblockno %in% c('28B', '28C') &
           blockno == i) %>%  
  group_by(blockno, cell.ntile) %>%
  # as_Spatial() %>% 
  subsample.distance(size = 45, d = 300) %>% #change size = 45
  st_as_sf() %>% 
  ungroup()

ts_site_list[[i]] <- cellqt
}

# Combine rows of each separate list to create single sf
ts_sites_sf <- bind_rows(ts_site_list)

# Find the centroid of each hex cell to create point/site location coordinates
ts_sites_pointqt_sf <- st_centroid(ts_sites_sf)

# Convert geometry to latitude and longitude to create dataframe
ts_sites_df <- ts_sites_pointqt_sf %>%
        st_transform(crs = st_crs(4326)) %>%
  sfheaders::sf_to_df(fill = T) %>%
  select(c(zone, blockno, subblockno, cell.ntile, oid, x, y)) %>%
  dplyr::rename('latitude' = y,
                'longitude' = x)

# Quick summary check of random sites generate per block
# Note: Block 28 has limited data to generate 45 sites.
ts_sites_df %>%
  group_by(blockno) %>%
  summarise(n = n())

# Add site names using naming convention 'AB-sampyear-blockno-site.order'
ts_sites_df <- ts_sites_df %>% 
 group_by(blockno) %>% 
  mutate(site_no = row_number(),
         site = paste('AB', assess_year, blockno, site_no, sep = '-')) %>% 
 ungroup()

# Unset seed
set.seed(NULL)

```

```{r site plot}

# Quick plot to examine sites for chosen block
ts_sites_df %>% 
  filter(blockno == 5) %>%
ggplot(aes(x = longitude, y = latitude))+
  geom_point()+
  # geom_path(arrow = arrow(type = "closed", length = unit(0.05, "npc")), aes(longitude, latitude, color = angle))+
  geom_point(aes(x = max(longitude), y = median(latitude)), color = 'red')+
  geom_text(aes(label = site_no), size = 4, color = 'red')

```

# Combine all new sites for assessment year with reference sites retained from 2020 in single dataframe

```{r rejoin ts sites}
# Create dataframe of reference sites
ts_ref_sites_df <- time_swim_ref_sites %>%
 st_as_sf(coords = c("x", "y")) %>% 
  st_set_crs(st_crs(7855)) %>%
  st_transform(crs = st_crs(4326)) %>%
  sfheaders::sf_to_df(fill = T) %>%
  select(site.new, blockno, subblockno, oid, cell.ntile, sam.count, x, y, ref_site) %>% 
  dplyr::rename('latitude' = y,
                'longitude' = x,
                'site' = site.new) %>% 
  mutate(blockno = as.integer(blockno),
         cell.ntile = as.integer(cell.ntile))

ts_sites_df <- ts_sites_df %>% 
  select(site, blockno, subblockno, oid, cell.ntile, longitude, latitude)

ts_sites_final_df <- bind_rows(ts_ref_sites_df, ts_sites_df) %>% 
 arrange(blockno) %>% 
 mutate(ref_site = ifelse(is.na(ref_site), 'new', ref_site))

# ts_sites_final_df <- ts_sites_df

ts_sites_final_sf  <-  ts_sites_final_df %>% 
  st_as_sf(coords = c('longitude', 'latitude'), crs = st_crs(4326))

ts_sites_final_gpx <- ts_sites_final_df %>% 
 dplyr::rename('name' = 'site') %>% 
 select(c(name, longitude, latitude)) %>% 
 add_column(description = NA, 
            comment = NA)

```
# Export data
```{r export data}

# Export folder path

gis_folder_path <- paste(file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/GIS/SpatialLayers/TimeSwimLayers',
Sys.info()[["user"]]), sep = '')))

data_folder_path <- paste(file.path(paste(sprintf('C:/Users/%s/Dropbox (UTAS Research)/DiveFisheries/Abalone/FISdata/', Sys.info()[["user"]]), paste('FIS_TimedSwimSurveys', assess_year, sep = ''), sep = '')))

# Export geospatial package for GIS and KML for Google Earth

outname_point <- paste(gis_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, ".gpkg", sep = ""), sep = '')
outname_point <- paste(data_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, ".gpkg", sep = ""), sep = '')

st_write(ts_sites_final_sf, dsn = outname_point, layer = "pointqt", driver = "GPKG", append = F)

# Export KML for Google Earth

outname_point_kml <- paste(gis_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, ".kml", sep = ""), sep = '')
outname_point_kml <- paste(data_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, ".kml", sep = ""), sep = '')

st_write(ts_sites_final_sf, dsn = outname_point_kml, layer = "pointqt", driver = "kml", append = F) 
 
# Export Excel files 
write.xlsx(ts_sites_final_df, paste(gis_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, ".xlsx", sep = ""), sep = ''), sheetName = "Sheet1", colNames = TRUE, rowNames = TRUE, append = FALSE)
write.xlsx(ts_sites_final_df, paste(data_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, ".xlsx", sep = ""), sep = ''), sheetName = "Sheet1", colNames = TRUE, rowNames = TRUE, append = FALSE)


# Export Excel file for GPX conversion
write.xlsx(ts_sites_final_gpx, paste(gis_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, '_GPX', ".xlsx", sep = ""), sep = ''), sheetName = "Sheet1", colNames = TRUE, rowNames = TRUE, append = FALSE)
write.xlsx(ts_sites_final_gpx, paste(data_folder_path, paste('/TimedSwimSites_EastCoast_Final_', assess_year, '_GPX', ".xlsx", sep = ""), sep = ''), sheetName = "Sheet1", colNames = TRUE, rowNames = TRUE, append = FALSE)
  
```