3.7 Data: Size & dimensions & creation
- Data comes in different formes
- Often in dataframe/matrix/table form
- A table/dataframe/matrix has the dimensions m (rows) by n (columns)
- Q: What are the data size measures? GB, MB, Byte etc.?
- In R you can easily create datasets of different sizes (try adapting different elements)…
# install.packages("pacman")
pacman::p_load(readr,
dplyr)
vector_length <- 10000 # Try 100000000
vector <- sample(1:10, vector_length, replace = TRUE)
n.col <- 4
n.rows <- length(vector)/n.col
data <- matrix(vector,
nrow = n.rows,
ncol = n.col) %>% data.frame()
write_csv(data, "./data/lab_outputs/data_artificial_10000.csv")
# Put this in a function if you like!…a more complicated example (were the data looks a bit more realistic)..
# install.packages("pacman")
pacman::p_load(randomNames,
readr,
dplyr)
n.rows <- 100000
data <- data.frame(id = 1:n.rows,
first.name = randomNames(1:n.rows, which.names="first"),
last.name = randomNames(1:n.rows, which.names="last"),
age = sample(15:90, n.rows, replace = T),
income = sample(500:3000, n.rows, replace = T)
)
write_csv(data, "./Data & material/lab_outputs/data_artificial_100000.csv") # 2.7 MB
# How many rows (m) and columns (n) does that dataset have?
n.rows <- 1000000
data <- data.frame(id = 1:n.rows,
first.name = randomNames(1:n.rows, which.names="first"),
last.name = randomNames(1:n.rows, which.names="last"),
age = sample(15:90, n.rows, replace = T),
income = sample(500:3000, n.rows, replace = T)
)
write_csv(data, "./Data & material/lab_outputs/data_artificial_1000000.csv") # 28 MB
# How many rows (m) and columns (n) does that dataset have?
n.rows <- 10000000
data <- data.frame(id = 1:n.rows,
first.name = randomNames(1:n.rows, which.names="first"),
last.name = randomNames(1:n.rows, which.names="last"),
age = sample(15:90, n.rows, replace = T),
income = sample(500:3000, n.rows, replace = T)
)
write_csv(data, "data_artificial_10000000.csv") # 294 MB
# You wanna get stuck?
# Try setting n.row below
# n.rows <- 100000000