* lightening/aarch64-cpu.c (SWPAL, CASAL): New instructions.
(swap_atomic, cas_atomic): Use better instructions if we have LSE.
* lightening/aarch64.c (get_hwcap, jit_get_cpu): Arrange to detect LSE
availability on GNU/Linux and Darwin.
Based on a patch by Tony Garnock-Jones. Thanks!
The problem with callr is that the register that contains the
function to be called, can be overwritten by the logic that moves
the values into argument registers. To fix this, I added a
get_callr_temp function that should return a platform specific
register that is not used to pass arguments. For Aarch64/Arm the
link registers seems to work; for Amd64/i686 the RAX register.
The function/tmp pair becomes an additional argument to the
parallel assigment; this way the original function register is not
accidentally overwritten.
The problem with calli is that it may not have enough temp
registers to move arguments. The windmill paper says that at most
one temporary register is needed for the parallel assignment.
However, we also need a temp register for mem-to-mem moves. So it
seems that we need a second temporary. For Amd64/i686 we have
only one temporary GPR and one temporary FPR. To fix this, I
modified the algorithm from the paper a bit: we perform the
mem-to-mem moves before the other moves. Later when we need the
temp to break cycles, there shouldn't be any mem-to-mem moves
left. So we should never need two temps at the same time.
* lightening/lightening.c: (get_callr_temp): New function; need
for each platform.
(prepare_call_args): Include the function/callr_temp pair in the
arguments for the parallel assignment.
* lightening/x86.c, lightening/arm.c, lightening/aarch64.c
(get_callr_temp): Implementation for each platform.
* lightening/arm.c (next_abi_arg): Fix the stack size for doubles.
* tests/call_10_2.c, tests/callr_10.c: New tests.
* tests/regarrays.inc: New file. Common code between the above two
tests that would be tedious to duplicate.
* lightening/x86.c (jit_try_shorten): If the address is within the
last instruction, don't shorten. If the intstruction is a jump, we
could elide it entirely in some cases, but we don't know if the user
captured the PC before calling jit_patch_here. Better to leave this
to the user.
Thanks to Helmut Eller for the bug report and test case in
https://gitlab.com/wingo/lightening/-/issues/17.
* lightening/lightening.c (abi_gpr_to_mem): Write whole words when
spilling GPRs to the stack. Always correct given that all Lightening
operations that write GPRs write the whole register, and the current
ABI targets allow writing the extra words. Closes#15.
Prevents useless over-alignment for ARM.
* lightening/lightening.c (struct jit_literal_pool_entry): Value is a
uintptr_t.
(emit_uintptr): New helper.
(emit_abs_reloc): Use new helper.
(patch_pending_literal): Value is a uintptr_t.
(emit_literal_pool): Adapt to literal entry being uintptr_t.
* lightening.h:
* lightening/lightening.c (jit_begin_data): Add max data size
parameter. If nonzero, can allow the JIT to avoid prematurely
emitting a constant pool.
(jit_end_data): Allow pending literals.
* tests/jmp_table.c (run_test): Use new API.
* lightening.h:
* lightening/lightening.c (jit_begin_data, jit_end_data)
(jit_emit_u8, jit_emit_u16, jit_emit_u32, jit_emit_u64): Add new raw
data-emitting primitives, bracketed by begin/end so that we can flush
constant pools first, if needed.
* lightening/lightening.c (struct jit_state): Add new emitting_data
flag.
(jit_begin, jit_reset, jit_end): Handle the new flag.
(emit_abs_reloc): Move here, from x86.c.
* lightening/x86.c (emit_abs_reloc): Remove.
(jit_try_shorten): Don't shorten if loc == start; could be raw data.
* tests/jmp_table.c: New test.
* lightening/arm-cpu.c (rotate_left): Fix the case of rotating by zero,
which produced undefined behavior. Many thanks to Andrew
Gierth (andrew at tao11 riddles org uk) for the debugging and the
fix.
It is unlikely for any ARM code to be close enough to not have needed
a veneer, but it is possible, especially if running in a program with
another JIT library.
* lightening/x86.c (reset_abi_arg_iterator): Reserve 32 stack bytes on
64-bit Windows systems, in accordance with ABI. Thanks a million to
Charles Stanhope for the patch and to Mike Gran for testing!
The existing calli / callr interface is for ABI calls. Sometimes though
you want to call some of your own code, just to get the current return
address. ARM's branch-and-link instructions are ideal for this but they
don't exist on x86; there we emulate them by adding corresponding
pop_link_register / push_link_register instructions that are no-ops on
ARM.
* lightening.h (FOR_EACH_INSTRUCTION): Add jit_jmpi_with_link,
pop_link_register, push_link_register.
* lightening/arm-cpu.c:
* lightening/x86-cpu.c:
* lightening/aarch64-cpu.c (jmpi_with_link, push_link_register)
(pop_link_register): Add implementations.
* lightening/arm.h:
* lightening/aarch64.h:
* lightening/x86.h (JIT_LR): New definition.
* tests/link-register.c: New test.
These operations emit the same code that GCC does for corresponding
operations under the sequential consistency memory model. It would be
possible to relax to acquire/release or something in the future.
