Skip to content

[WIP] Improve windows support #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 68 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
fe58529
Use correct data type in unix
auterium Dec 25, 2019
4693cf6
Merge branch 'master' of https://github.com/bluejekyll/pg-extend-rs
auterium Dec 26, 2019
3feffa9
Merge branch 'master' of https://github.com/bluejekyll/pg-extend-rs
auterium Dec 26, 2019
e55b2d8
fixup build script to properly support features
bluejekyll Dec 27, 2019
1cdfccd
cleanup Makefile, reduce duplicate definitions
bluejekyll Dec 27, 2019
0fab9ea
Merge branch 'master' of https://github.com/bluejekyll/pg-extend-rs
auterium Dec 27, 2019
095dc80
Merge branch 'improve-test-feature-execution' of https://github.com/b…
auterium Dec 27, 2019
5a164c4
revert sgjmp changes
bluejekyll Dec 27, 2019
9090d45
test dedupped environment variables
bluejekyll Dec 27, 2019
9acc1c9
use decoded variables
bluejekyll Dec 27, 2019
1dbce10
cleanliness should wait for postgres builds
bluejekyll Dec 27, 2019
78a19ea
Merge branch 'improve-test-feature-execution' of https://github.com/b…
auterium Dec 27, 2019
3a7513f
restart PG between tests
bluejekyll Dec 27, 2019
7e6c36f
remove deps-cache, was too large to cache
bluejekyll Dec 27, 2019
66b329e
check connectivity after start
bluejekyll Dec 27, 2019
a07e53a
try to delay after file copies in tests
bluejekyll Dec 27, 2019
dfc4913
add some logging information on failure
bluejekyll Dec 27, 2019
4247c44
add extra debug info to pg
bluejekyll Dec 27, 2019
b6db880
ensure all tests are single threaded against PG
bluejekyll Dec 28, 2019
eef7e02
only run the adding_tests, narrowing down the issue on linux
bluejekyll Dec 28, 2019
a70ffd9
reenable logging tests
bluejekyll Dec 28, 2019
7899ef6
double check that logging is the issue
bluejekyll Dec 28, 2019
7a21080
stop copying file to tmp
bluejekyll Dec 28, 2019
fc92097
remove cargo cache from CI, not working
bluejekyll Dec 28, 2019
54e096f
see if panicking tests also cause issues
bluejekyll Dec 28, 2019
60ba457
retest with logging enabled
bluejekyll Dec 28, 2019
1178224
Merge branch 'improve-test-feature-execution' of https://github.com/b…
auterium Dec 28, 2019
26fc01d
add back all tests
bluejekyll Dec 28, 2019
86b1537
remove parallel restrictions
bluejekyll Dec 29, 2019
f661b0d
change clippy to do entire workspace in one go
bluejekyll Dec 29, 2019
1c635d4
remove sleeps
bluejekyll Dec 29, 2019
2da1a60
Merge branch 'improve-test-feature-execution' of https://github.com/b…
auterium Dec 29, 2019
776921e
Add array support
auterium Dec 29, 2019
fb5bb60
Example for array sum
auterium Dec 29, 2019
8501f81
Fix array support
auterium Dec 29, 2019
581de6d
Remove dependency on slice_cast crate
auterium Dec 29, 2019
254f518
Merge branch 'master' of https://github.com/bluejekyll/pg-extend-rs i…
auterium Dec 29, 2019
d2f380f
Merge branch 'master' of https://github.com/bluejekyll/pg-extend-rs
auterium Dec 29, 2019
2f21d21
DRY
auterium Dec 29, 2019
b66204a
Improve docs wording & hide try_cast from docs
auterium Dec 29, 2019
f816edf
Limit slices support to primitive numbers only
auterium Dec 29, 2019
02a97da
Revert wrapper
auterium Dec 29, 2019
9478c88
Add back missing assertion
auterium Dec 29, 2019
8f0d58e
Fix missplaced assertion
auterium Dec 29, 2019
f394ec1
Fix indentation
auterium Dec 29, 2019
14e3c60
Comments & doc hidding of new trait
auterium Dec 29, 2019
f6c0463
Set bool_ type for macos
auterium Dec 30, 2019
a465218
Revert type override
auterium Dec 30, 2019
a95580f
Try conditional override for MacOS on PG12
auterium Dec 30, 2019
9046014
Init with pgbool!()
auterium Dec 30, 2019
632b90a
bool_ type for pg11 in macos
auterium Dec 30, 2019
e6d7bed
Set bool_ type for unix familty
auterium Dec 30, 2019
6a47166
Make clippy happy
auterium Dec 30, 2019
ec56a44
Cargo fmt
auterium Dec 30, 2019
59f45d6
Remove bogus implementation
auterium Dec 30, 2019
360c479
DetoastedArrayWrapper with drop for memory free
auterium Dec 30, 2019
5a75777
Make clippy happy
auterium Dec 30, 2019
ecda157
Fix format
auterium Dec 30, 2019
0c13ccb
Proper generation of create statements with arrays
auterium Dec 30, 2019
fa59df4
Integration tests for array sums
auterium Dec 31, 2019
b594ec2
Cargo fmt
auterium Dec 31, 2019
ef1b691
Use stdbool.h
auterium Dec 31, 2019
349c2c0
Improve create stmt generation
auterium Dec 31, 2019
a2ea1c9
Fix arrays support for Windows builds
auterium Jan 3, 2020
ad4fea3
Merge branch 'master' of https://github.com/bluejekyll/pg-extend-rs
auterium Jan 3, 2020
dc42a13
Merge branch 'master' into improve-windows-support
auterium Jan 3, 2020
0a6ec37
Soundness
auterium Jan 3, 2020
97ab100
Rust fmt
auterium Jan 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .cargo/config
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,8 @@
[target.'cfg(unix)']
rustflags = "-C link-arg=-undefineddynamic_lookup"

# Uncomment the appropiate line depending on your PG version
[target.'cfg(windows)']
rustflags = "-C link-arg=/FORCE"
#rustflags = "-C link-arg=/FORCE:UNRESOLVED -C link-arg=C:/PROGRA~1/POSTGR~1/10/lib/postgres.lib"
#rustflags = "-C link-arg=/FORCE:UNRESOLVED -C link-arg=C:/PROGRA~1/POSTGR~1/11/lib/postgres.lib"
#rustflags = "-C link-arg=/FORCE:UNRESOLVED -C link-arg=C:/PROGRA~1/POSTGR~1/12/lib/postgres.lib"
15 changes: 11 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,21 @@ First install Postgres. The build should be able to find the directory for the P

`PG_INCLUDE_PATH=[/path/to/postgres]/include/server # e.g. /usr/local/pgsql/include/server`

For the dynamic library to compile, your project should also have `.cargo/config` file with content:
For the dynamic library to compile, your project should also have a `.cargo/config` file. The contents of this file varies based on your platform.

### POSIX (aka `unix` family)
```toml
[target.'cfg(unix)']
[build]
rustflags = "-C link-arg=-undefineddynamic_lookup"
```

This informs the linker that some of the symbols for Postgres won't be available until runtime on the dynamic library load.

[target.'cfg(windows)']
rustflags = "-C link-arg=/FORCE"
### Windows
Building for Windows is somewhat cumbersome as the MSVC linker requires you to set the location of `postgres.lib`, but this will depend both on your installation path as well as your PG version. Here's an example where the installation path is the default one for PG 12:
```toml
[build]
rustflags = "-C link-arg=/FORCE:UNRESOLVED -C link-arg=C:/PROGRA~1/POSTGR~1/12/lib/postgres.lib"
```

This informs the linker that some of the symbols for Postgres won't be available until runtime on the dynamic library load.
Expand Down
2 changes: 1 addition & 1 deletion examples/adding/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ fn sum_float_array(arr: &[f32]) -> f32 {
arr.iter().sum()
}

// Test array of f32
// Test array of f64
#[pg_extern]
fn sum_double_array(arr: &[f64]) -> f64 {
arr.iter().sum()
Expand Down
4 changes: 2 additions & 2 deletions pg-extend/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use std::process::Command;
fn main() {
let out_path = PathBuf::from(env::var("OUT_DIR").unwrap()).join("postgres.rs");

let pg_config = env::var("PG_CONFIG").unwrap_or_else(|_| "pg_config".to_string());
let pg_config = env::var("PG_CONFIG").unwrap_or("pg_config".to_string());

// Re-run this if wrapper.h changes
println!("cargo:rerun-if-changed=wrapper.h");
Expand Down Expand Up @@ -163,7 +163,7 @@ fn include_dir(pg_config: &str) -> Result<String, env::VarError> {
env::var("PG_INCLUDE_PATH").or_else(|err| {
match Command::new(pg_config).arg("--includedir-server").output() {
Ok(out) => Ok(String::from_utf8(out.stdout).unwrap().trim().to_string()),
Err(..) => Err(err),
Err(_) => Err(err),
}
})
}
Expand Down
15 changes: 11 additions & 4 deletions pg-extend/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -273,13 +273,20 @@ macro_rules! pg_create_stmt_bin {
#[cfg(target_os = "macos")]
const DYLIB_EXT: &str = "dylib";

#[cfg(target_os = "windows")]
const DYLIB_EXT: &str = "dll";
#[cfg(unix)]
fn main() {
let lib_name = env!("CARGO_PKG_NAME").replace("-", "_");

let lib_path = env::args().nth(1).unwrap_or(format!("target/release/lib{}.{}", lib_name, DYLIB_EXT));

$( println!("{}", lib::$func(&lib_path)); )*
}

#[cfg(windows)]
fn main() {
const LIB_NAME: &str = env!("CARGO_PKG_NAME");
let lib_name = env!("CARGO_PKG_NAME").replace("-", "_");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When building, the resulting files will always use underscores, so we need to replace dashes with underscores, otherwise there's a mismatch


let lib_path = env::args().nth(1).unwrap_or_else(|| format!("target/release/lib{}.{}", LIB_NAME, DYLIB_EXT));
let lib_path = env::args().nth(1).unwrap_or(format!("target\\\\release\\\\{}.dll", lib_name));

$( println!("{}", lib::$func(&lib_path)); )*
}
Expand Down
40 changes: 40 additions & 0 deletions pg-extend/src/log.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
//! [`pg_log!`]: ../macro.pg_log.html
//! [`Level` enum]: enum.Level.html

#[cfg(not(windows))]
use std::ffi::CString;
use std::fmt;
use std::os::raw::{c_char, c_int};
Expand Down Expand Up @@ -97,6 +98,29 @@ pub enum Level {
Panic = pg_sys::PANIC as isize,
}

impl std::fmt::Display for Level {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let level = match self {
Level::Debug5 => "DEBUG5",
Level::Debug4 => "DEBUG4",
Level::Debug3 => "DEBUG3",
Level::Debug2 => "DEBUG2",
Level::Debug1 => "DEBUG1",
Level::Log => "LOG",
#[cfg(not(feature = "postgres-9"))]
Level::LogServerOnly => "LOG_SERVER_ONLY",
Level::Info => "INFO",
Level::Notice => "NOTICE",
Level::Warning => "WARNING",
Level::Error => "ERROR",
Level::Fatal => "FATAL",
Level::Panic => "PANIC",
};

write!(f, "{}", level)
}
}

impl From<Level> for c_int {
fn from(level: Level) -> Self {
level as isize as c_int
Expand Down Expand Up @@ -207,6 +231,7 @@ macro_rules! pg_log {

// WARNING: this is not part of the crate's public API and is subject to change at any time
#[doc(hidden)]
#[cfg(not(windows))]
pub fn __private_api_log(
args: fmt::Arguments,
level: Level,
Expand Down Expand Up @@ -242,3 +267,18 @@ pub fn __private_api_log(
}
}
}

// TODO: improve logging and figure out a way to deal with side-effects
#[doc(hidden)]
#[cfg(windows)]
pub fn __private_api_log(
args: fmt::Arguments,
level: Level,
&(_module_path, _file, _line): &(*const c_char, *const c_char, u32),
) {
eprintln!("{}: {}", level, args);

if level as usize >= Level::Error as usize {
panic!("");
}
}
79 changes: 38 additions & 41 deletions pg-extend/src/pg_datum.rs
Original file line number Diff line number Diff line change
Expand Up @@ -371,14 +371,14 @@ struct DetoastedArrayWrapper {
}

impl DetoastedArrayWrapper {
unsafe fn detoasted(datum: Datum) -> Result<Self, &'static str> {
fn detoasted(datum: Datum) -> Result<Self, &'static str> {
let datum = datum as *mut pg_sys::varlena;
if datum.is_null() {
return Err("datum was NULL");
}

#[allow(clippy::cast_ptr_alignment)]
let arr_type = pg_sys::pg_detoast_datum(datum) as *mut pg_sys::ArrayType;
let arr_type = unsafe { pg_sys::pg_detoast_datum(datum) as *mut pg_sys::ArrayType };

#[allow(clippy::cast_ptr_alignment)]
let original_datum = datum as *mut pg_sys::ArrayType;
Expand All @@ -395,16 +395,16 @@ impl DetoastedArrayWrapper {
impl Drop for DetoastedArrayWrapper {
fn drop(&mut self) {
if self.arr_type != self.original_datum {
unsafe {
if !self.arr_type.is_null() {
pg_sys::pfree(self.arr_type as *mut _);
}
if !self.elements.is_null() {
pg_sys::pfree(self.elements as *mut _);
}
if !self.nulls.is_null() {
pg_sys::pfree(self.nulls as *mut _);
}
if !self.arr_type.is_null() {
unsafe { pg_sys::pfree(self.arr_type as *mut _) }
}

if !self.elements.is_null() {
unsafe { pg_sys::pfree(self.elements as *mut _) }
}

if !self.nulls.is_null() {
unsafe { pg_sys::pfree(self.nulls as *mut _) }
}
}
}
Expand All @@ -430,26 +430,25 @@ where
'mc: 's,
{
if let Some(datum) = datum.0 {
unsafe {
let mut detoasted_wrapper = DetoastedArrayWrapper::detoasted(datum)?;
let mut detoasted_wrapper = DetoastedArrayWrapper::detoasted(datum)?;

if (*(detoasted_wrapper.arr_type)).ndim > 1 {
return Err("argument must be empty or one-dimensional array");
}
if unsafe { (*(detoasted_wrapper.arr_type)).ndim } > 1 {
return Err("argument must be empty or one-dimensional array");
}

let mut elmlen: pg_sys::int16 = 0;
let mut elmbyval = pgbool!(false);
let mut elmalign: ::std::os::raw::c_char = 0;
let mut elmlen: pg_sys::int16 = 0;
let mut elmbyval = false;
let mut elmalign: c_char = 0;
let mut nelems = 0;

let datums = unsafe {
pg_sys::get_typlenbyvalalign(
(*(detoasted_wrapper.arr_type)).elemtype,
&mut elmlen,
&mut elmbyval,
&mut elmalign,
);

let mut nelems: i32 = 0;

pg_sys::deconstruct_array(
detoasted_wrapper.arr_type,
(*(detoasted_wrapper.arr_type)).elemtype,
Expand All @@ -461,27 +460,25 @@ where
&mut nelems,
);

let datums = std::slice::from_raw_parts(
std::slice::from_raw_parts(
detoasted_wrapper.elements as *const Datum,
nelems as usize,
);

// This is where the conversion from `&[Datum]` is done to `&[T]` by a simple type casting,
// however, we should use `T::try_cast(&'mc PgAllocator, Datum)` to ignore nulls
let mem_size_datums = std::mem::size_of_val(datums);
let datums = if mem_size_datums == 0 {
std::slice::from_raw_parts(datums.as_ptr() as *const T, 0)
} else {
let mem_size_type = std::mem::size_of::<T>();
assert_eq!(mem_size_datums % mem_size_type, 0);
std::slice::from_raw_parts(
datums.as_ptr() as *const T,
mem_size_datums / mem_size_type,
)
};

Ok(datums)
}
)
};

// This is where the conversion from `&[Datum]` is done to `&[T]` by a simple type casting
let mem_size_datums = std::mem::size_of_val(datums);
let datums = if mem_size_datums == 0 {
unsafe { std::slice::from_raw_parts(datums.as_ptr() as *const T, 0) }
} else {
let mem_size_type = std::mem::size_of::<T>();
assert_eq!(mem_size_datums % mem_size_type, 0);
let len = mem_size_datums / mem_size_type;

unsafe { std::slice::from_raw_parts(datums.as_ptr() as *const T, len) }
};

Ok(datums)
} else {
Err("datum was NULL")
}
Expand Down
7 changes: 2 additions & 5 deletions pg-extern-attr/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,8 @@ fn sql_function_options(arg_types: &[Type]) -> TokenStream {
quote!(
{
let optional_args = [ #( <#arg_types>::is_option() ),* ];
if optional_args.iter().all(|&x| x) { "" }
else if !optional_args.iter().any(|&x| x) { " STRICT" }
else {
panic!("Cannot mix Option and non-Option arguments.");
}
if !optional_args.iter().any(|&x| x) { " STRICT" }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows for mixing Option with non-Option arguments.

else { "" }
},
)
}
Expand Down