Compare commits

...

8 Commits

Author SHA1 Message Date
CPunch d015eec5f1 updated CONTRIB 2022-09-01 20:15:35 -05:00
CPunch cf01657cc2 fixes for winblows 2022-09-01 20:07:29 -05:00
CPunch 587d9a26e5 lpolllist.c: fix poll ifdef paths 2022-09-01 20:04:54 -05:00
CPunch b23057b219 Refactoring: reorganized files 2022-09-01 20:00:37 -05:00
CPunch 169313ee39 shell: migrated to new vector API
- removed array API from lmem.h
- sterm.c: basically left as-is. see source for notes as to why
2022-09-01 19:35:52 -05:00
CPunch 44086f563b cnc: migrate to new vector API 2022-09-01 19:12:13 -05:00
CPunch 13398dbdf6 lsocket.[ch] & lpeer.c: migrated to new vector API 2022-09-01 19:05:56 -05:00
CPunch af09e74263 lmem.h: new laikaM_*Vector macros
- these will slowly replace laikaM_*array
- lpeer.[ch] has been migrated
2022-09-01 18:47:29 -05:00
53 changed files with 842 additions and 742 deletions

View File

@ -16,8 +16,8 @@ Looking for some simple tasks that need to get done for that sweet 'contributor'
- Change `lib/lin/linshell.c` to use openpty() instead of forkpty() for BSD support - Change `lib/lin/linshell.c` to use openpty() instead of forkpty() for BSD support
- Fix address sanitizer for CMake DEBUG builds - Fix address sanitizer for CMake DEBUG builds
- Change laikaT_getTime in `lib/src/ltask.c` to not use C11 features - Change laikaT_getTime in `lib/src/core/ltask.c` to not use C11 features
- Implement more LAIKA_BOX_* VMs in `lib/include/lbox.h` - Implement more LAIKA_BOX_* VMs in `lib/include/core/lbox.h`
- Import more WinAPI manually using the method listed below - Import more WinAPI manually using the method listed below
## Bot: Windows API Imports Obfuscation ## Bot: Windows API Imports Obfuscation
@ -63,7 +63,7 @@ If the `real` & `hashed` match, that means our manual runtime import and the imp
Now just replace all of the calls to the raw WinAPI (in our case, ShellExecuteA) with our new manually imported oShellExecuteA function pointer. Format & commit your changes, and open a PR and I'll merge your changes. Thanks! Now just replace all of the calls to the raw WinAPI (in our case, ShellExecuteA) with our new manually imported oShellExecuteA function pointer. Format & commit your changes, and open a PR and I'll merge your changes. Thanks!
## Lib: Error Handling ## Lib: Error Handling
Error handling in Laika is done via the 'lerror.h' header library. It's a small and simple error handling solution written for laika, however can be stripped and used as a simple error handling library. Error handling in Laika is used similarly to other languages, implementing a try & catch block and is achieved using setjmp(). The LAIKA_ERROR(...) is used to throw errors. Error handling in Laika is done via the 'lib/core/lerror.h' header library. It's a small and simple error handling solution written for laika, however can be stripped and used as a simple error handling library. Error handling in Laika is used similarly to other languages, implementing a try & catch block and is achieved using setjmp() & longjmp(). The LAIKA_ERROR(...) macro is used to throw errors.
Example: Example:
```C ```C
@ -84,15 +84,13 @@ Some minor inconveniences include:
- not thread safe. - not thread safe.
## Lib: Packet Handlers ## Lib: Packet Handlers
Laika has a simple binary protocol & a small backend (see `lib/src/net/lpeer.c`) to handle packets to/from peers. `lib/include/net/lpacket.h` includes descriptions for each packet type. For an example of proper packet handler definitions see `bot/src/bot.c`. It boils down to passing a sLaika_peerPacketInfo table to laikaS_newPeer. To add packet handlers to the bot, add your handler info to laikaB_pktTbl in `bot/src/bot.c`. To add packet handlers to the shell, add your handler info to shellC_pktTbl in `shell/src/sclient.c`. For adding packet handlers to cnc, make sure you add them to the corresponding table in `cnc/src/cnc.c`, laikaC_botPktTbl for packets being received from a bot peer, laikaC_authPktTbl for packets being received from an auth peer (shell), or DEFAULT_PKT_TBL if it's received by all peer types (things like handshakes, keep-alive, etc.)
Laika has a simple binary protocol & a small backend (see `lib/src/lpeer.c`) to handle packets to/from peers. `lib/include/lpacket.h` includes descriptions for each packet type. For an example of proper packet handler definitions see `bot/src/bot.c`. It boils down to passing a sLaika_peerPacketInfo table to laikaS_newPeer. To add packet handlers to the bot, add your handler info to laikaB_pktTbl in `bot/src/bot.c`. To add packet handlers to the shell, add your handler info to shellC_pktTbl in `shell/src/sclient.c`. For adding packet handlers to cnc, make sure you add them to the corresponding table in `cnc/src/cnc.c`, laikaC_botPktTbl for packets being received from a bot peer, laikaC_authPktTbl for packets being received from an auth peer (shell), or DEFAULT_PKT_TBL if it's received by all peer types (things like handshakes, keep-alive, etc.)
## Lib: Task Service ## Lib: Task Service
Tasks can be scheduled on a delta-period (call X function every approximate N seconds). laikaT_pollTasks() is used to check & run any currently queued tasks. This is useful for sending keep-alive packets, polling shell pipes, or other repeatably scheduled tasks. Most laikaT_pollTasks() calls are done in the peerHandler for each client/server. Tasks can be scheduled on a delta-period (call X function every approximate N seconds). laikaT_pollTasks() is used to check & run any currently queued tasks. This is useful for sending keep-alive packets, polling shell pipes, or other repeatably scheduled tasks. Most laikaT_pollTasks() calls are done in the peerHandler for each client/server.
## Lib: VM Boxes ## Lib: VM Boxes
Laika has a tiny VM for decrypting sensitive information. For details on the ISA read `lib/include/lvm.h`, for information on how to use them read `lib/include/lbox.h`. Feel free to write your own boxes and contribute them :D Laika has a tiny VM for decrypting sensitive information. For details on the ISA read `lib/include/core/lvm.h`, for information on how to use them read `lib/include/core/lbox.h`. Feel free to write your own boxes and contribute them :D
## Bot: Platform-specific backends ## Bot: Platform-specific backends
`bot/win` and `bot/lin` include code for platform-specific code that can't be quickly "ifdef"d away. These mainly include stuff like persistence or opening pseudo-ttys. `bot/win` and `bot/lin` include code for platform-specific code that can't be quickly "ifdef"d away. These mainly include stuff like persistence or opening pseudo-ttys.

View File

@ -1,13 +1,13 @@
#ifndef LAIKA_BOT_H #ifndef LAIKA_BOT_H
#define LAIKA_BOT_H #define LAIKA_BOT_H
#include "core/lsodium.h"
#include "core/ltask.h"
#include "laika.h" #include "laika.h"
#include "lpacket.h" #include "net/lpacket.h"
#include "lpeer.h" #include "net/lpeer.h"
#include "lpolllist.h" #include "net/lpolllist.h"
#include "lsocket.h" #include "net/lsocket.h"
#include "lsodium.h"
#include "ltask.h"
struct sLaika_shell; struct sLaika_shell;
struct sLaika_bot struct sLaika_bot

View File

@ -2,7 +2,7 @@
#define LAIKA_SHELL_H #define LAIKA_SHELL_H
#include "laika.h" #include "laika.h"
#include "lpacket.h" #include "net/lpacket.h"
#include <time.h> #include <time.h>

View File

@ -1,10 +1,10 @@
/* platform specific code for achieving persistence on linux */ /* platform specific code for achieving persistence on linux */
#include "lbox.h" #include "core/lbox.h"
#include "core/lerror.h"
#include "core/lmem.h"
#include "lconfig.h" #include "lconfig.h"
#include "lerror.h" #include "net/lsocket.h"
#include "lmem.h"
#include "lsocket.h"
#include "persist.h" #include "persist.h"
#include <pwd.h> #include <pwd.h>

View File

@ -1,9 +1,9 @@
/* platform specific code for opening shells in linux */ /* platform specific code for opening shells in linux */
#include "bot.h" #include "bot.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
#include "ltask.h" #include "core/ltask.h"
#include "shell.h" #include "shell.h"
#include <pty.h> #include <pty.h>

View File

@ -1,9 +1,9 @@
#include "bot.h" #include "bot.h"
#include "lbox.h" #include "core/lbox.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
#include "lsodium.h" #include "core/lsodium.h"
#include "shell.h" #include "shell.h"
void laikaB_handleHandshakeResponse(struct sLaika_peer *peer, LAIKAPKT_SIZE sz, void *uData) void laikaB_handleHandshakeResponse(struct sLaika_peer *peer, LAIKAPKT_SIZE sz, void *uData)

View File

@ -1,8 +1,8 @@
#include "bot.h" #include "bot.h"
#include "lbox.h" #include "core/lbox.h"
#include "core/lerror.h"
#include "core/ltask.h"
#include "lconfig.h" #include "lconfig.h"
#include "lerror.h"
#include "ltask.h"
#include "lobf.h" #include "lobf.h"
#include "persist.h" #include "persist.h"
#include "shell.h" #include "shell.h"

View File

@ -1,8 +1,8 @@
#include "shell.h" #include "shell.h"
#include "bot.h" #include "bot.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
#include <inttypes.h> #include <inttypes.h>
#include <stdio.h> #include <stdio.h>

View File

@ -6,11 +6,11 @@
#pragma comment(lib, "Shlwapi.lib") #pragma comment(lib, "Shlwapi.lib")
#include "lbox.h" #include "core/lbox.h"
#include "core/lerror.h"
#include "core/lmem.h"
#include "core/lvm.h"
#include "lconfig.h" #include "lconfig.h"
#include "lerror.h"
#include "lmem.h"
#include "lvm.h"
#include "lobf.h" #include "lobf.h"
#include "persist.h" #include "persist.h"

View File

@ -1,8 +1,8 @@
/* platform specific code for opening shells (pseudo consoles) on windows */ /* platform specific code for opening shells (pseudo consoles) on windows */
#include "bot.h" #include "bot.h"
#include "core/lerror.h"
#include "core/lmem.h"
#include "lobf.h" #include "lobf.h"
#include "lerror.h"
#include "lmem.h"
#include "shell.h" #include "shell.h"
#include <process.h> #include <process.h>

View File

@ -1,13 +1,14 @@
#ifndef LAIKA_CNC_H #ifndef LAIKA_CNC_H
#define LAIKA_CNC_H #define LAIKA_CNC_H
#include "hashmap.h" #include "core/hashmap.h"
#include "core/lmem.h"
#include "core/ltask.h"
#include "laika.h" #include "laika.h"
#include "lpacket.h" #include "net/lpacket.h"
#include "lpeer.h" #include "net/lpeer.h"
#include "lpolllist.h" #include "net/lpolllist.h"
#include "lsocket.h" #include "net/lsocket.h"
#include "ltask.h"
/* kill peers if they haven't ping'd within a minute */ /* kill peers if they haven't ping'd within a minute */
#define LAIKA_PEER_TIMEOUT 60 * 1000 #define LAIKA_PEER_TIMEOUT 60 * 1000
@ -20,12 +21,8 @@ struct sLaika_cnc
struct sLaika_socket sock; struct sLaika_socket sock;
struct sLaika_pollList pList; struct sLaika_pollList pList;
struct hashmap *peers; /* holds all peers, lookup using pubkey */ struct hashmap *peers; /* holds all peers, lookup using pubkey */
struct sLaika_peer **authPeers; /* holds connected panel peers */ laikaM_newVector(struct sLaika_peer *, authPeers); /* holds connected panel peers */
uint8_t **authKeys; laikaM_newVector(uint8_t *, authKeys);
int authKeysCount;
int authKeysCap;
int authPeersCount;
int authPeersCap;
uint16_t port; uint16_t port;
}; };

View File

@ -2,7 +2,7 @@
#define LAIKA_CNC_PANEL_H #define LAIKA_CNC_PANEL_H
#include "cnc.h" #include "cnc.h"
#include "lpeer.h" #include "net/lpeer.h"
void laikaC_sendPeerList(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer); void laikaC_sendPeerList(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer);
void laikaC_sendNewPeer(struct sLaika_peer *authPeer, struct sLaika_peer *bot); void laikaC_sendNewPeer(struct sLaika_peer *authPeer, struct sLaika_peer *bot);

View File

@ -2,10 +2,10 @@
#define LAIKA_CNC_PEER_H #define LAIKA_CNC_PEER_H
#include "laika.h" #include "laika.h"
#include "lpacket.h" #include "net/lpacket.h"
#include "lpeer.h" #include "net/lpeer.h"
#include "lpolllist.h" #include "net/lpolllist.h"
#include "lsocket.h" #include "net/lsocket.h"
struct sLaika_peerInfo struct sLaika_peerInfo
{ {

View File

@ -1,12 +1,12 @@
#include "cnc.h" #include "cnc.h"
#include "core/lerror.h"
#include "core/lmem.h"
#include "core/lsodium.h"
#include "core/ltask.h"
#include "cpanel.h" #include "cpanel.h"
#include "cpeer.h" #include "cpeer.h"
#include "lerror.h" #include "net/lsocket.h"
#include "lmem.h"
#include "lsocket.h"
#include "lsodium.h"
#include "ltask.h"
/* ======================================[[ PeerHashMap ]]======================================= */ /* ======================================[[ PeerHashMap ]]======================================= */
@ -161,12 +161,8 @@ struct sLaika_cnc *laikaC_newCNC(uint16_t port)
/* init peer hashmap & panel list */ /* init peer hashmap & panel list */
cnc->peers = hashmap_new(sizeof(tCNC_PeerHashElem), 8, 0, 0, cnc_PeerElemHash, cnc->peers = hashmap_new(sizeof(tCNC_PeerHashElem), 8, 0, 0, cnc_PeerElemHash,
cnc_PeerElemCompare, NULL, NULL); cnc_PeerElemCompare, NULL, NULL);
cnc->authPeers = NULL; laikaM_initVector(cnc->authPeers, 4);
cnc->authKeys = NULL; laikaM_initVector(cnc->authKeys, 4);
cnc->authKeysCount = 0;
cnc->authKeysCap = 4;
cnc->authPeersCap = 4;
cnc->authPeersCount = 0;
cnc->port = port; cnc->port = port;
/* init socket (we just need it for the raw socket fd and abstracted API :P) & pollList */ /* init socket (we just need it for the raw socket fd and abstracted API :P) & pollList */
@ -207,7 +203,7 @@ void laikaC_freeCNC(struct sLaika_cnc *cnc)
hashmap_free(cnc->peers); hashmap_free(cnc->peers);
/* free auth keys */ /* free auth keys */
for (i = 0; i < cnc->authKeysCount; i++) { for (i = 0; i < laikaM_countVector(cnc->authKeys); i++) {
laikaM_free(cnc->authKeys[i]); laikaM_free(cnc->authKeys[i]);
} }
laikaM_free(cnc->authKeys); laikaM_free(cnc->authKeys);
@ -222,7 +218,7 @@ void laikaC_onAddPeer(struct sLaika_cnc *cnc, struct sLaika_peer *peer)
hashmap_set(cnc->peers, &(tCNC_PeerHashElem){.pub = peer->peerPub, .peer = peer}); hashmap_set(cnc->peers, &(tCNC_PeerHashElem){.pub = peer->peerPub, .peer = peer});
/* notify connected panels of the newly connected peer */ /* notify connected panels of the newly connected peer */
for (i = 0; i < cnc->authPeersCount; i++) { for (i = 0; i < laikaM_countVector(cnc->authPeers); i++) {
laikaC_sendNewPeer(cnc->authPeers[i], peer); laikaC_sendNewPeer(cnc->authPeers[i], peer);
} }
@ -273,7 +269,7 @@ void laikaC_onRmvPeer(struct sLaika_cnc *cnc, struct sLaika_peer *peer)
} }
/* notify connected panels of the disconnected peer */ /* notify connected panels of the disconnected peer */
for (i = 0; i < cnc->authPeersCount; i++) { for (i = 0; i < laikaM_countVector(cnc->authPeers); i++) {
laikaC_sendRmvPeer(cnc->authPeers[i], peer); laikaC_sendRmvPeer(cnc->authPeers[i], peer);
} }
@ -316,11 +312,10 @@ void laikaC_setPeerType(struct sLaika_cnc *cnc, struct sLaika_peer *peer, PEERTY
void laikaC_addAuth(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer) void laikaC_addAuth(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer)
{ {
/* grow array if we need to */ /* grow array if we need to */
laikaM_growarray(struct sLaika_peer *, cnc->authPeers, 1, cnc->authPeersCount, laikaM_growVector(struct sLaika_peer *, cnc->authPeers, 1);
cnc->authPeersCap);
/* insert into authenticated peer table */ /* insert into authenticated peer table */
cnc->authPeers[cnc->authPeersCount++] = authPeer; cnc->authPeers[laikaM_countVector(cnc->authPeers)++] = authPeer;
LAIKA_DEBUG("added panel %p!\n", authPeer); LAIKA_DEBUG("added panel %p!\n", authPeer);
} }
@ -329,9 +324,9 @@ void laikaC_rmvAuth(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer)
{ {
int i; int i;
for (i = 0; i < cnc->authPeersCount; i++) { for (i = 0; i < laikaM_countVector(cnc->authPeers); i++) {
if (cnc->authPeers[i] == authPeer) { /* we found the index for our panel! */ if (cnc->authPeers[i] == authPeer) { /* we found the index for our panel! */
laikaM_rmvarray(cnc->authPeers, cnc->authPeersCount, i, 1); laikaM_rmvVector(cnc->authPeers, i, 1);
return; return;
} }
} }
@ -340,14 +335,14 @@ void laikaC_rmvAuth(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer)
void laikaC_addAuthKey(struct sLaika_cnc *cnc, const char *key) void laikaC_addAuthKey(struct sLaika_cnc *cnc, const char *key)
{ {
uint8_t *buf; uint8_t *buf;
laikaM_growarray(uint8_t *, cnc->authKeys, 1, cnc->authKeysCount, cnc->authKeysCap); laikaM_growVector(uint8_t *, cnc->authKeys, 1);
buf = laikaM_malloc(crypto_kx_PUBLICKEYBYTES); buf = laikaM_malloc(crypto_kx_PUBLICKEYBYTES);
if (!laikaK_loadKeys(buf, NULL, key, NULL)) if (!laikaK_loadKeys(buf, NULL, key, NULL))
LAIKA_ERROR("Failed to load key '%s'\n", key); LAIKA_ERROR("Failed to load key '%s'\n", key);
/* insert key */ /* insert key */
cnc->authKeys[cnc->authKeysCount++] = buf; cnc->authKeys[laikaM_countVector(cnc->authKeys)++] = buf;
printf("[~] Added authenticated public key '%s'\n", key); printf("[~] Added authenticated public key '%s'\n", key);
} }
@ -455,7 +450,7 @@ bool laikaC_iterPeersNext(struct sLaika_cnc *cnc, size_t *i, struct sLaika_peer
{ {
tCNC_PeerHashElem *elem; tCNC_PeerHashElem *elem;
if (hashmap_iter(cnc->peers, i, (void *)&elem)) { if (hashmap_iter(cnc->peers, i, (void **)&elem)) {
*peer = elem->peer; *peer = elem->peer;
return true; return true;
} }

View File

@ -1,9 +1,9 @@
#include "cpanel.h" #include "cpanel.h"
#include "cnc.h" #include "cnc.h"
#include "core/lerror.h"
#include "core/lmem.h"
#include "cpeer.h" #include "cpeer.h"
#include "lerror.h"
#include "lmem.h"
void laikaC_sendPeerList(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer) void laikaC_sendPeerList(struct sLaika_cnc *cnc, struct sLaika_peer *authPeer)
{ {

View File

@ -1,8 +1,8 @@
#include "cpeer.h" #include "cpeer.h"
#include "cnc.h" #include "cnc.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
/* =======================================[[ Peer Info ]]======================================= */ /* =======================================[[ Peer Info ]]======================================= */
@ -155,7 +155,7 @@ void laikaC_handlePeerLoginReq(struct sLaika_peer *peer, LAIKAPKT_SIZE sz, void
break; break;
case PEER_AUTH: case PEER_AUTH:
/* check that peer's pubkey is authenticated */ /* check that peer's pubkey is authenticated */
if (!laikaK_checkAuth(peer->peerPub, cnc->authKeys, cnc->authKeysCount)) if (!laikaK_checkAuth(peer->peerPub, cnc->authKeys, laikaM_countVector(cnc->authKeys)))
LAIKA_ERROR("laikaC_handlePeerHandshake: Unauthorized panel!\n"); LAIKA_ERROR("laikaC_handlePeerHandshake: Unauthorized panel!\n");
LAIKA_DEBUG("Accepted authenticated panel %p\n", peer); LAIKA_DEBUG("Accepted authenticated panel %p\n", peer);

View File

@ -1,7 +1,7 @@
#include "cnc.h" #include "cnc.h"
#include "ini.h" #include "core/ini.h"
#include "core/ltask.h"
#include "lconfig.h" #include "lconfig.h"
#include "ltask.h"
#include <stdio.h> #include <stdio.h>

View File

@ -8,7 +8,7 @@ project(LaikaLib VERSION ${LAIKA_VERSION_MAJOR}.${LAIKA_VERSION_MINOR})
set_property(GLOBAL PROPERTY USE_FOLDERS ON) set_property(GLOBAL PROPERTY USE_FOLDERS ON)
# compile LaikaLib library # compile LaikaLib library
file(GLOB_RECURSE LIBSOURCE ${CMAKE_CURRENT_SOURCE_DIR}/src/**.c ${CMAKE_CURRENT_SOURCE_DIR}/vendor/**.c) file(GLOB_RECURSE LIBSOURCE ${CMAKE_CURRENT_SOURCE_DIR}/src/**.c)
file(GLOB_RECURSE LIBHEADERS ${CMAKE_CURRENT_SOURCE_DIR}/include/**.h) file(GLOB_RECURSE LIBHEADERS ${CMAKE_CURRENT_SOURCE_DIR}/include/**.h)
# include platform specific backends # include platform specific backends

View File

@ -16,7 +16,8 @@ https://github.com/benhoyt/inih
/* Make this header file easier to include in C++ code */ /* Make this header file easier to include in C++ code */
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C"
{
#endif #endif
#include <stdio.h> #include <stdio.h>
@ -28,12 +29,10 @@ extern "C" {
/* Typedef for prototype of handler function. */ /* Typedef for prototype of handler function. */
#if INI_HANDLER_LINENO #if INI_HANDLER_LINENO
typedef int (*ini_handler)(void* user, const char* section, typedef int (*ini_handler)(void *user, const char *section, const char *name, const char *value,
const char* name, const char* value,
int lineno); int lineno);
#else #else
typedef int (*ini_handler)(void* user, const char* section, typedef int (*ini_handler)(void *user, const char *section, const char *name, const char *value);
const char* name, const char* value);
#endif #endif
/* Typedef for prototype of fgets-style reader function. */ /* Typedef for prototype of fgets-style reader function. */
@ -61,8 +60,7 @@ int ini_parse_file(FILE* file, ini_handler handler, void* user);
/* Same as ini_parse(), but takes an ini_reader function pointer instead of /* Same as ini_parse(), but takes an ini_reader function pointer instead of
filename. Used for implementing custom or string-based I/O (see also filename. Used for implementing custom or string-based I/O (see also
ini_parse_string). */ ini_parse_string). */
int ini_parse_stream(ini_reader reader, void* stream, ini_handler handler, int ini_parse_stream(ini_reader reader, void *stream, ini_handler handler, void *user);
void* user);
/* Same as ini_parse(), but takes a zero-terminated string with the INI data /* Same as ini_parse(), but takes a zero-terminated string with the INI data
instead of a file. Useful for parsing INI data from a network socket or instead of a file. Useful for parsing INI data from a network socket or
@ -149,7 +147,6 @@ int ini_parse_string(const char* string, ini_handler handler, void* user);
# define INI_CUSTOM_ALLOCATOR 0 # define INI_CUSTOM_ALLOCATOR 0
#endif #endif
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

View File

@ -1,10 +1,10 @@
#ifndef LAIKA_BOX_H #ifndef LAIKA_BOX_H
#define LAIKA_BOX_H #define LAIKA_BOX_H
#include "core/lmem.h"
#include "core/lsodium.h"
#include "core/lvm.h"
#include "laika.h" #include "laika.h"
#include "lmem.h"
#include "lsodium.h"
#include "lvm.h"
#include <inttypes.h> #include <inttypes.h>

60
lib/include/core/lmem.h Normal file
View File

@ -0,0 +1,60 @@
#ifndef LAIKA_MEM_H
#define LAIKA_MEM_H
#include "laika.h"
#define GROW_FACTOR 2
/* microsoft strikes again with their lack of support for VLAs */
#if _MSC_VER
# define VLA(type, var, sz) type *var = laikaM_malloc(sizeof(type) * sz);
# define ENDVLA(var) laikaM_free(var);
#else
# define VLA(type, var, sz) type var[sz];
# define ENDVLA(var) ((void)0) /* no op */
#endif
#define laikaM_malloc(sz) laikaM_realloc(NULL, sz)
#define laikaM_free(buf) laikaM_realloc(buf, 0)
/* ========================================[[ Vectors ]]======================================== */
#define laikaM_countVector(name) name##_COUNT
#define laikaM_capVector(name) name##_CAP
#define laikaM_newVector(type, name) \
type *name; \
int name##_COUNT; \
int name##_CAP
#define laikaM_initVector(name, startCap) \
name = NULL; \
name##_COUNT = 0; \
name##_CAP = startCap
#define laikaM_growVector(type, name, needed) \
if (name##_COUNT + needed >= name##_CAP || name == NULL) { \
name##_CAP = (name##_CAP + needed) * GROW_FACTOR; \
name = (type *)laikaM_realloc(name, sizeof(type) * name##_CAP); \
}
/* moves vector elements above indx down by numElem, removing numElem elements at indx */
#define laikaM_rmvVector(name, indx, numElem) \
do { \
int _i, _sz = ((name##_COUNT - indx) - numElem); \
for (_i = 0; _i < _sz; _i++) \
name[indx + _i] = name[indx + numElem + _i]; \
name##_COUNT -= numElem; \
} while (0);
/* moves vector elements above indx up by numElem, inserting numElem elements at indx */
#define laikaM_insertVector(name, indx, numElem) \
do { \
int _i; \
for (_i = name##_COUNT; _i > indx; _i--) \
name[_i] = name[_i - 1]; \
name##_COUNT += numElem; \
} while (0);
void *laikaM_realloc(void *buf, size_t sz);
#endif

View File

@ -9,8 +9,8 @@
fit this specific use case. fit this specific use case.
*/ */
#include "core/lerror.h"
#include "laika.h" #include "laika.h"
#include "lerror.h"
#include <inttypes.h> #include <inttypes.h>

View File

@ -1,46 +0,0 @@
#ifndef LAIKA_MEM_H
#define LAIKA_MEM_H
#include "laika.h"
#define GROW_FACTOR 2
/* microsoft strikes again with their lack of support for VLAs */
#if _MSC_VER
# define VLA(type, var, sz) type *var = laikaM_malloc(sizeof(type) * sz);
# define ENDVLA(var) laikaM_free(var);
#else
# define VLA(type, var, sz) type var[sz];
# define ENDVLA(var) ((void)0) /* no op */
#endif
#define laikaM_malloc(sz) laikaM_realloc(NULL, sz)
#define laikaM_free(buf) laikaM_realloc(buf, 0)
#define laikaM_growarray(type, buf, needed, count, capacity) \
if (count + needed >= capacity || buf == NULL) { \
capacity = (capacity + needed) * GROW_FACTOR; \
buf = (type *)laikaM_realloc(buf, sizeof(type) * capacity); \
}
/* moves array elements above indx down by numElem, removing numElem elements at indx */
#define laikaM_rmvarray(buf, count, indx, numElem) \
do { \
int _i, _sz = ((count - indx) - numElem); \
for (_i = 0; _i < _sz; _i++) \
buf[indx + _i] = buf[indx + numElem + _i]; \
count -= numElem; \
} while (0);
/* moves array elements above indx up by numElem, inserting numElem elements at indx */
#define laikaM_insertarray(buf, count, indx, numElem) \
do { \
int _i; \
for (_i = count; _i > indx; _i--) \
buf[_i] = buf[_i - 1]; \
count += numElem; \
} while (0);
void *laikaM_realloc(void *buf, size_t sz);
#endif

View File

@ -1,11 +1,11 @@
#ifndef LAIKA_PEER_H #ifndef LAIKA_PEER_H
#define LAIKA_PEER_H #define LAIKA_PEER_H
#include "core/lsodium.h"
#include "laika.h" #include "laika.h"
#include "lpacket.h" #include "net/lpacket.h"
#include "lpolllist.h" #include "net/lpolllist.h"
#include "lsocket.h" #include "net/lsocket.h"
#include "lsodium.h"
typedef enum typedef enum
{ {

View File

@ -1,9 +1,10 @@
#ifndef LAIKA_POLLLIST_H #ifndef LAIKA_POLLLIST_H
#define LAIKA_POLLLIST_H #define LAIKA_POLLLIST_H
#include "hashmap.h" #include "core/hashmap.h"
#include "core/lmem.h"
#include "laika.h" #include "laika.h"
#include "lsocket.h" #include "net/lsocket.h"
#include <stdbool.h> #include <stdbool.h>
@ -20,22 +21,17 @@ struct sLaika_pollEvent
struct sLaika_pollList struct sLaika_pollList
{ {
struct hashmap *sockets; struct hashmap *sockets;
struct sLaika_socket **outQueue; /* holds sockets which have data needed to be sent */ /* holds sockets which have data needed to be sent */
struct sLaika_pollEvent *revents; laikaM_newVector(struct sLaika_socket *, outQueue);
laikaM_newVector(struct sLaika_pollEvent, revents);
#ifdef LAIKA_USE_EPOLL #ifdef LAIKA_USE_EPOLL
/* epoll */ /* epoll */
struct epoll_event ev, ep_events[MAX_EPOLL_EVENTS]; struct epoll_event ev, ep_events[MAX_EPOLL_EVENTS];
SOCKET epollfd; SOCKET epollfd;
#else #else
/* raw poll descriptor */ /* raw poll descriptor */
PollFD *fds; laikaM_newVector(PollFD, fds);
int fdCapacity;
int fdCount;
#endif #endif
int reventCap;
int reventCount;
int outCap;
int outCount;
}; };
void laikaP_initPList(struct sLaika_pollList *pList); void laikaP_initPList(struct sLaika_pollList *pList);

View File

@ -54,7 +54,8 @@ typedef void buffer_t;
# define SOCKETERROR(x) (x == -1) # define SOCKETERROR(x) (x == -1)
#endif #endif
#include "laika.h" #include "laika.h"
#include "lsodium.h" #include "core/lsodium.h"
#include "core/lmem.h"
#include <fcntl.h> #include <fcntl.h>
#include <stdbool.h> #include <stdbool.h>
@ -79,12 +80,8 @@ struct sLaika_socket
pollEvent onPollIn; pollEvent onPollIn;
pollEvent onPollOut; pollEvent onPollOut;
void *uData; /* passed to onPollFail */ void *uData; /* passed to onPollFail */
uint8_t *outBuf; /* raw data to be sent() */ laikaM_newVector(uint8_t, outBuf); /* raw data to be sent() */
uint8_t *inBuf; /* raw data we recv()'d */ laikaM_newVector(uint8_t, inBuf); /* raw data we recv()'d */
int outCount;
int inCount;
int outCap;
int inCap;
bool flipEndian; bool flipEndian;
bool setPollOut; /* is EPOLLOUT/POLLOUT is set on sock's pollfd ? */ bool setPollOut; /* is EPOLLOUT/POLLOUT is set on sock's pollfd ? */
}; };

View File

@ -2,7 +2,7 @@
// Use of this source code is governed by an MIT-style // Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file. // license that can be found in the LICENSE file.
#include "hashmap.h" #include "core/hashmap.h"
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <stdint.h>
@ -23,18 +23,21 @@ void hashmap_set_allocator(void *(*malloc)(size_t), void (*free)(void*))
_free = free; _free = free;
} }
#define panic(_msg_) { \ #define panic(_msg_) \
{ \
fprintf(stderr, "panic: %s (%s:%d)\n", (_msg_), __FILE__, __LINE__); \ fprintf(stderr, "panic: %s (%s:%d)\n", (_msg_), __FILE__, __LINE__); \
exit(1); \ exit(1); \
} }
struct bucket { struct bucket
{
uint64_t hash : 48; uint64_t hash : 48;
uint64_t dib : 16; uint64_t dib : 16;
}; };
// hashmap is an open addressed hash map using robinhood hashing. // hashmap is an open addressed hash map using robinhood hashing.
struct hashmap { struct hashmap
{
void *(*malloc)(size_t); void *(*malloc)(size_t);
void *(*realloc)(void *, size_t); void *(*realloc)(void *, size_t);
void (*free)(void *); void (*free)(void *);
@ -58,32 +61,30 @@ struct hashmap {
void *edata; void *edata;
}; };
static struct bucket *bucket_at(struct hashmap *map, size_t index) { static struct bucket *bucket_at(struct hashmap *map, size_t index)
{
return (struct bucket *)(((char *)map->buckets) + (map->bucketsz * index)); return (struct bucket *)(((char *)map->buckets) + (map->bucketsz * index));
} }
static void *bucket_item(struct bucket *entry) { static void *bucket_item(struct bucket *entry)
{
return ((char *)entry) + sizeof(struct bucket); return ((char *)entry) + sizeof(struct bucket);
} }
static uint64_t get_hash(struct hashmap *map, const void *key) { static uint64_t get_hash(struct hashmap *map, const void *key)
{
return map->hash(key, map->seed0, map->seed1) << 16 >> 16; return map->hash(key, map->seed0, map->seed1) << 16 >> 16;
} }
// hashmap_new_with_allocator returns a new hash map using a custom allocator. // hashmap_new_with_allocator returns a new hash map using a custom allocator.
// See hashmap_new for more information information // See hashmap_new for more information information
struct hashmap *hashmap_new_with_allocator( struct hashmap *
void *(*_malloc)(size_t), hashmap_new_with_allocator(void *(*_malloc)(size_t), void *(*_realloc)(void *, size_t),
void *(*_realloc)(void*, size_t), void (*_free)(void *), size_t elsize, size_t cap, uint64_t seed0,
void (*_free)(void*), uint64_t seed1,
size_t elsize, size_t cap, uint64_t (*hash)(const void *item, uint64_t seed0, uint64_t seed1),
uint64_t seed0, uint64_t seed1, int (*compare)(const void *a, const void *b, void *udata),
uint64_t (*hash)(const void *item, void (*elfree)(void *item), void *udata)
uint64_t seed0, uint64_t seed1),
int (*compare)(const void *a, const void *b,
void *udata),
void (*elfree)(void *item),
void *udata)
{ {
_malloc = _malloc ? _malloc : malloc; _malloc = _malloc ? _malloc : malloc;
_realloc = _realloc ? _realloc : realloc; _realloc = _realloc ? _realloc : realloc;
@ -135,7 +136,6 @@ struct hashmap *hashmap_new_with_allocator(
return map; return map;
} }
// hashmap_new returns a new hash map. // hashmap_new returns a new hash map.
// Param `elsize` is the size of each element in the tree. Every element that // Param `elsize` is the size of each element in the tree. Every element that
// is inserted, deleted, or retrieved will be this size. // is inserted, deleted, or retrieved will be this size.
@ -153,40 +153,35 @@ struct hashmap *hashmap_new_with_allocator(
// The hashmap must be freed with hashmap_free(). // The hashmap must be freed with hashmap_free().
// Param `elfree` is a function that frees a specific item. This should be NULL // Param `elfree` is a function that frees a specific item. This should be NULL
// unless you're storing some kind of reference data in the hash. // unless you're storing some kind of reference data in the hash.
struct hashmap *hashmap_new(size_t elsize, size_t cap, struct hashmap *hashmap_new(size_t elsize, size_t cap, uint64_t seed0, uint64_t seed1,
uint64_t seed0, uint64_t seed1, uint64_t (*hash)(const void *item, uint64_t seed0, uint64_t seed1),
uint64_t (*hash)(const void *item, int (*compare)(const void *a, const void *b, void *udata),
uint64_t seed0, uint64_t seed1), void (*elfree)(void *item), void *udata)
int (*compare)(const void *a, const void *b,
void *udata),
void (*elfree)(void *item),
void *udata)
{ {
return hashmap_new_with_allocator( return hashmap_new_with_allocator((_malloc ? _malloc : malloc), (_realloc ? _realloc : realloc),
(_malloc?_malloc:malloc), (_free ? _free : free), elsize, cap, seed0, seed1, hash,
(_realloc?_realloc:realloc), compare, elfree, udata);
(_free?_free:free),
elsize, cap, seed0, seed1, hash, compare, elfree, udata
);
} }
static void free_elements(struct hashmap *map) { static void free_elements(struct hashmap *map)
{
if (map->elfree) { if (map->elfree) {
for (size_t i = 0; i < map->nbuckets; i++) { for (size_t i = 0; i < map->nbuckets; i++) {
struct bucket *bucket = bucket_at(map, i); struct bucket *bucket = bucket_at(map, i);
if (bucket->dib) map->elfree(bucket_item(bucket)); if (bucket->dib)
map->elfree(bucket_item(bucket));
} }
} }
} }
// hashmap_clear quickly clears the map. // hashmap_clear quickly clears the map.
// Every item is called with the element-freeing function given in hashmap_new, // Every item is called with the element-freeing function given in hashmap_new,
// if present, to free any data referenced in the elements of the hashmap. // if present, to free any data referenced in the elements of the hashmap.
// When the update_cap is provided, the map's capacity will be updated to match // When the update_cap is provided, the map's capacity will be updated to match
// the currently number of allocated buckets. This is an optimization to ensure // the currently number of allocated buckets. This is an optimization to ensure
// that this operation does not perform any allocations. // that this operation does not perform any allocations.
void hashmap_clear(struct hashmap *map, bool update_cap) { void hashmap_clear(struct hashmap *map, bool update_cap)
{
map->count = 0; map->count = 0;
free_elements(map); free_elements(map);
if (update_cap) { if (update_cap) {
@ -205,11 +200,10 @@ void hashmap_clear(struct hashmap *map, bool update_cap) {
map->shrinkat = map->nbuckets * 0.10; map->shrinkat = map->nbuckets * 0.10;
} }
static bool resize(struct hashmap *map, size_t new_cap)
static bool resize(struct hashmap *map, size_t new_cap) { {
struct hashmap *map2 = hashmap_new(map->elsize, new_cap, map->seed1, struct hashmap *map2 = hashmap_new(map->elsize, new_cap, map->seed1, map->seed1, map->hash,
map->seed1, map->hash, map->compare, map->compare, map->elfree, map->udata);
map->elfree, map->udata);
if (!map2) { if (!map2) {
return false; return false;
} }
@ -249,7 +243,8 @@ static bool resize(struct hashmap *map, size_t new_cap) {
// replaced then it is returned otherwise NULL is returned. This operation // replaced then it is returned otherwise NULL is returned. This operation
// may allocate memory. If the system is unable to allocate additional // may allocate memory. If the system is unable to allocate additional
// memory then NULL is returned and hashmap_oom() returns true. // memory then NULL is returned and hashmap_oom() returns true.
void *hashmap_set(struct hashmap *map, const void *item) { void *hashmap_set(struct hashmap *map, const void *item)
{
if (!item) { if (!item) {
panic("item is null"); panic("item is null");
} }
@ -261,7 +256,6 @@ void *hashmap_set(struct hashmap *map, const void *item) {
} }
} }
struct bucket *entry = map->edata; struct bucket *entry = map->edata;
entry->hash = get_hash(map, item); entry->hash = get_hash(map, item);
entry->dib = 1; entry->dib = 1;
@ -276,9 +270,7 @@ void *hashmap_set(struct hashmap *map, const void *item) {
return NULL; return NULL;
} }
if (entry->hash == bucket->hash && if (entry->hash == bucket->hash &&
map->compare(bucket_item(entry), bucket_item(bucket), map->compare(bucket_item(entry), bucket_item(bucket), map->udata) == 0) {
map->udata) == 0)
{
memcpy(map->spare, bucket_item(bucket), map->elsize); memcpy(map->spare, bucket_item(bucket), map->elsize);
memcpy(bucket_item(bucket), bucket_item(entry), map->elsize); memcpy(bucket_item(bucket), bucket_item(entry), map->elsize);
return map->spare; return map->spare;
@ -295,7 +287,8 @@ void *hashmap_set(struct hashmap *map, const void *item) {
// hashmap_get returns the item based on the provided key. If the item is not // hashmap_get returns the item based on the provided key. If the item is not
// found then NULL is returned. // found then NULL is returned.
void *hashmap_get(struct hashmap *map, const void *key) { void *hashmap_get(struct hashmap *map, const void *key)
{
if (!key) { if (!key) {
panic("key is null"); panic("key is null");
} }
@ -306,9 +299,7 @@ void *hashmap_get(struct hashmap *map, const void *key) {
if (!bucket->dib) { if (!bucket->dib) {
return NULL; return NULL;
} }
if (bucket->hash == hash && if (bucket->hash == hash && map->compare(key, bucket_item(bucket), map->udata) == 0) {
map->compare(key, bucket_item(bucket), map->udata) == 0)
{
return bucket_item(bucket); return bucket_item(bucket);
} }
i = (i + 1) & map->mask; i = (i + 1) & map->mask;
@ -318,7 +309,8 @@ void *hashmap_get(struct hashmap *map, const void *key) {
// hashmap_probe returns the item in the bucket at position or NULL if an item // hashmap_probe returns the item in the bucket at position or NULL if an item
// is not set for that bucket. The position is 'moduloed' by the number of // is not set for that bucket. The position is 'moduloed' by the number of
// buckets in the hashmap. // buckets in the hashmap.
void *hashmap_probe(struct hashmap *map, uint64_t position) { void *hashmap_probe(struct hashmap *map, uint64_t position)
{
size_t i = position & map->mask; size_t i = position & map->mask;
struct bucket *bucket = bucket_at(map, i); struct bucket *bucket = bucket_at(map, i);
if (!bucket->dib) { if (!bucket->dib) {
@ -327,10 +319,10 @@ void *hashmap_probe(struct hashmap *map, uint64_t position) {
return bucket_item(bucket); return bucket_item(bucket);
} }
// hashmap_delete removes an item from the hash map and returns it. If the // hashmap_delete removes an item from the hash map and returns it. If the
// item is not found then NULL is returned. // item is not found then NULL is returned.
void *hashmap_delete(struct hashmap *map, void *key) { void *hashmap_delete(struct hashmap *map, void *key)
{
if (!key) { if (!key) {
panic("key is null"); panic("key is null");
} }
@ -342,9 +334,7 @@ void *hashmap_delete(struct hashmap *map, void *key) {
if (!bucket->dib) { if (!bucket->dib) {
return NULL; return NULL;
} }
if (bucket->hash == hash && if (bucket->hash == hash && map->compare(key, bucket_item(bucket), map->udata) == 0) {
map->compare(key, bucket_item(bucket), map->udata) == 0)
{
memcpy(map->spare, bucket_item(bucket), map->elsize); memcpy(map->spare, bucket_item(bucket), map->elsize);
bucket->dib = 0; bucket->dib = 0;
for (;;) { for (;;) {
@ -372,15 +362,18 @@ void *hashmap_delete(struct hashmap *map, void *key) {
} }
// hashmap_count returns the number of items in the hash map. // hashmap_count returns the number of items in the hash map.
size_t hashmap_count(struct hashmap *map) { size_t hashmap_count(struct hashmap *map)
{
return map->count; return map->count;
} }
// hashmap_free frees the hash map // hashmap_free frees the hash map
// Every item is called with the element-freeing function given in hashmap_new, // Every item is called with the element-freeing function given in hashmap_new,
// if present, to free any data referenced in the elements of the hashmap. // if present, to free any data referenced in the elements of the hashmap.
void hashmap_free(struct hashmap *map) { void hashmap_free(struct hashmap *map)
if (!map) return; {
if (!map)
return;
free_elements(map); free_elements(map);
map->free(map->buckets); map->free(map->buckets);
map->free(map); map->free(map);
@ -388,15 +381,15 @@ void hashmap_free(struct hashmap *map) {
// hashmap_oom returns true if the last hashmap_set() call failed due to the // hashmap_oom returns true if the last hashmap_set() call failed due to the
// system being out of memory. // system being out of memory.
bool hashmap_oom(struct hashmap *map) { bool hashmap_oom(struct hashmap *map)
{
return map->oom; return map->oom;
} }
// hashmap_scan iterates over all items in the hash map // hashmap_scan iterates over all items in the hash map
// Param `iter` can return false to stop iteration early. // Param `iter` can return false to stop iteration early.
// Returns false if the iteration has been stopped early. // Returns false if the iteration has been stopped early.
bool hashmap_scan(struct hashmap *map, bool hashmap_scan(struct hashmap *map, bool (*iter)(const void *item, void *udata), void *udata)
bool (*iter)(const void *item, void *udata), void *udata)
{ {
for (size_t i = 0; i < map->nbuckets; i++) { for (size_t i = 0; i < map->nbuckets; i++) {
struct bucket *bucket = bucket_at(map, i); struct bucket *bucket = bucket_at(map, i);
@ -409,7 +402,6 @@ bool hashmap_scan(struct hashmap *map,
return true; return true;
} }
// hashmap_iter iterates one key at a time yielding a reference to an // hashmap_iter iterates one key at a time yielding a reference to an
// entry at each iteration. Useful to write simple loops and avoid writing // entry at each iteration. Useful to write simple loops and avoid writing
// dedicated callbacks and udata structures, as in hashmap_scan. // dedicated callbacks and udata structures, as in hashmap_scan.
@ -433,7 +425,8 @@ bool hashmap_iter(struct hashmap *map, size_t *i, void **item)
struct bucket *bucket; struct bucket *bucket;
do { do {
if (*i >= map->nbuckets) return false; if (*i >= map->nbuckets)
return false;
bucket = bucket_at(map, *i); bucket = bucket_at(map, *i);
(*i)++; (*i)++;
@ -444,7 +437,6 @@ bool hashmap_iter(struct hashmap *map, size_t *i, void **item)
return true; return true;
} }
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
// SipHash reference C implementation // SipHash reference C implementation
// //
@ -462,32 +454,42 @@ bool hashmap_iter(struct hashmap *map, size_t *i, void **item)
// //
// default: SipHash-2-4 // default: SipHash-2-4
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
static uint64_t SIP64(const uint8_t *in, const size_t inlen, static uint64_t SIP64(const uint8_t *in, const size_t inlen, uint64_t seed0, uint64_t seed1)
uint64_t seed0, uint64_t seed1)
{ {
#define U8TO64_LE(p) \ #define U8TO64_LE(p) \
{ (((uint64_t)((p)[0])) | ((uint64_t)((p)[1]) << 8) | \ {(((uint64_t)((p)[0])) | ((uint64_t)((p)[1]) << 8) | ((uint64_t)((p)[2]) << 16) | \
((uint64_t)((p)[2]) << 16) | ((uint64_t)((p)[3]) << 24) | \ ((uint64_t)((p)[3]) << 24) | ((uint64_t)((p)[4]) << 32) | ((uint64_t)((p)[5]) << 40) | \
((uint64_t)((p)[4]) << 32) | ((uint64_t)((p)[5]) << 40) | \
((uint64_t)((p)[6]) << 48) | ((uint64_t)((p)[7]) << 56))} ((uint64_t)((p)[6]) << 48) | ((uint64_t)((p)[7]) << 56))}
#define U64TO8_LE(p, v) \ #define U64TO8_LE(p, v) \
{ U32TO8_LE((p), (uint32_t)((v))); \ { \
U32TO8_LE((p) + 4, (uint32_t)((v) >> 32)); } U32TO8_LE((p), (uint32_t)((v))); \
U32TO8_LE((p) + 4, (uint32_t)((v) >> 32)); \
}
#define U32TO8_LE(p, v) \ #define U32TO8_LE(p, v) \
{ (p)[0] = (uint8_t)((v)); \ { \
(p)[0] = (uint8_t)((v)); \
(p)[1] = (uint8_t)((v) >> 8); \ (p)[1] = (uint8_t)((v) >> 8); \
(p)[2] = (uint8_t)((v) >> 16); \ (p)[2] = (uint8_t)((v) >> 16); \
(p)[3] = (uint8_t)((v) >> 24); } (p)[3] = (uint8_t)((v) >> 24); \
}
#define ROTL(x, b) (uint64_t)(((x) << (b)) | ((x) >> (64 - (b)))) #define ROTL(x, b) (uint64_t)(((x) << (b)) | ((x) >> (64 - (b))))
#define SIPROUND \ #define SIPROUND \
{ v0 += v1; v1 = ROTL(v1, 13); \ { \
v1 ^= v0; v0 = ROTL(v0, 32); \ v0 += v1; \
v2 += v3; v3 = ROTL(v3, 16); \ v1 = ROTL(v1, 13); \
v1 ^= v0; \
v0 = ROTL(v0, 32); \
v2 += v3; \
v3 = ROTL(v3, 16); \
v3 ^= v2; \ v3 ^= v2; \
v0 += v3; v3 = ROTL(v3, 21); \ v0 += v3; \
v3 = ROTL(v3, 21); \
v3 ^= v0; \ v3 ^= v0; \
v2 += v1; v1 = ROTL(v1, 17); \ v2 += v1; \
v1 ^= v2; v2 = ROTL(v2, 32); } v1 = ROTL(v1, 17); \
v1 ^= v2; \
v2 = ROTL(v2, 32); \
}
uint64_t k0 = U8TO64_LE((uint8_t *)&seed0); uint64_t k0 = U8TO64_LE((uint8_t *)&seed0);
uint64_t k1 = U8TO64_LE((uint8_t *)&seed1); uint64_t k1 = U8TO64_LE((uint8_t *)&seed1);
uint64_t v3 = UINT64_C(0x7465646279746573) ^ k1; uint64_t v3 = UINT64_C(0x7465646279746573) ^ k1;
@ -498,26 +500,40 @@ static uint64_t SIP64(const uint8_t *in, const size_t inlen,
for (; in != end; in += 8) { for (; in != end; in += 8) {
uint64_t m = U8TO64_LE(in); uint64_t m = U8TO64_LE(in);
v3 ^= m; v3 ^= m;
SIPROUND; SIPROUND; SIPROUND;
SIPROUND;
v0 ^= m; v0 ^= m;
} }
const int left = inlen & 7; const int left = inlen & 7;
uint64_t b = ((uint64_t)inlen) << 56; uint64_t b = ((uint64_t)inlen) << 56;
switch (left) { switch (left) {
case 7: b |= ((uint64_t)in[6]) << 48; case 7:
case 6: b |= ((uint64_t)in[5]) << 40; b |= ((uint64_t)in[6]) << 48;
case 5: b |= ((uint64_t)in[4]) << 32; case 6:
case 4: b |= ((uint64_t)in[3]) << 24; b |= ((uint64_t)in[5]) << 40;
case 3: b |= ((uint64_t)in[2]) << 16; case 5:
case 2: b |= ((uint64_t)in[1]) << 8; b |= ((uint64_t)in[4]) << 32;
case 1: b |= ((uint64_t)in[0]); break; case 4:
case 0: break; b |= ((uint64_t)in[3]) << 24;
case 3:
b |= ((uint64_t)in[2]) << 16;
case 2:
b |= ((uint64_t)in[1]) << 8;
case 1:
b |= ((uint64_t)in[0]);
break;
case 0:
break;
} }
v3 ^= b; v3 ^= b;
SIPROUND; SIPROUND; SIPROUND;
SIPROUND;
v0 ^= b; v0 ^= b;
v2 ^= 0xff; v2 ^= 0xff;
SIPROUND; SIPROUND; SIPROUND; SIPROUND; SIPROUND;
SIPROUND;
SIPROUND;
SIPROUND;
b = v0 ^ v1 ^ v2 ^ v3; b = v0 ^ v1 ^ v2 ^ v3;
uint64_t out = 0; uint64_t out = 0;
U64TO8_LE((uint8_t *)&out, b); U64TO8_LE((uint8_t *)&out, b);
@ -530,9 +546,15 @@ static uint64_t SIP64(const uint8_t *in, const size_t inlen,
// //
// Murmur3_86_128 // Murmur3_86_128
//----------------------------------------------------------------------------- //-----------------------------------------------------------------------------
static void MM86128(const void *key, const int len, uint32_t seed, void *out) { static void MM86128(const void *key, const int len, uint32_t seed, void *out)
{
#define ROTL32(x, r) ((x << r) | (x >> (32 - r))) #define ROTL32(x, r) ((x << r) | (x >> (32 - r)))
#define FMIX32(h) h^=h>>16; h*=0x85ebca6b; h^=h>>13; h*=0xc2b2ae35; h^=h>>16; #define FMIX32(h) \
h ^= h >> 16; \
h *= 0x85ebca6b; \
h ^= h >> 13; \
h *= 0xc2b2ae35; \
h ^= h >> 16;
const uint8_t *data = (const uint8_t *)key; const uint8_t *data = (const uint8_t *)key;
const int nblocks = len / 16; const int nblocks = len / 16;
uint32_t h1 = seed; uint32_t h1 = seed;
@ -549,14 +571,34 @@ static void MM86128(const void *key, const int len, uint32_t seed, void *out) {
uint32_t k2 = blocks[i * 4 + 1]; uint32_t k2 = blocks[i * 4 + 1];
uint32_t k3 = blocks[i * 4 + 2]; uint32_t k3 = blocks[i * 4 + 2];
uint32_t k4 = blocks[i * 4 + 3]; uint32_t k4 = blocks[i * 4 + 3];
k1 *= c1; k1 = ROTL32(k1,15); k1 *= c2; h1 ^= k1; k1 *= c1;
h1 = ROTL32(h1,19); h1 += h2; h1 = h1*5+0x561ccd1b; k1 = ROTL32(k1, 15);
k2 *= c2; k2 = ROTL32(k2,16); k2 *= c3; h2 ^= k2; k1 *= c2;
h2 = ROTL32(h2,17); h2 += h3; h2 = h2*5+0x0bcaa747; h1 ^= k1;
k3 *= c3; k3 = ROTL32(k3,17); k3 *= c4; h3 ^= k3; h1 = ROTL32(h1, 19);
h3 = ROTL32(h3,15); h3 += h4; h3 = h3*5+0x96cd1c35; h1 += h2;
k4 *= c4; k4 = ROTL32(k4,18); k4 *= c1; h4 ^= k4; h1 = h1 * 5 + 0x561ccd1b;
h4 = ROTL32(h4,13); h4 += h1; h4 = h4*5+0x32ac3b17; k2 *= c2;
k2 = ROTL32(k2, 16);
k2 *= c3;
h2 ^= k2;
h2 = ROTL32(h2, 17);
h2 += h3;
h2 = h2 * 5 + 0x0bcaa747;
k3 *= c3;
k3 = ROTL32(k3, 17);
k3 *= c4;
h3 ^= k3;
h3 = ROTL32(h3, 15);
h3 += h4;
h3 = h3 * 5 + 0x96cd1c35;
k4 *= c4;
k4 = ROTL32(k4, 18);
k4 *= c1;
h4 ^= k4;
h4 = ROTL32(h4, 13);
h4 += h1;
h4 = h4 * 5 + 0x32ac3b17;
} }
const uint8_t *tail = (const uint8_t *)(data + nblocks * 16); const uint8_t *tail = (const uint8_t *)(data + nblocks * 16);
uint32_t k1 = 0; uint32_t k1 = 0;
@ -564,32 +606,73 @@ static void MM86128(const void *key, const int len, uint32_t seed, void *out) {
uint32_t k3 = 0; uint32_t k3 = 0;
uint32_t k4 = 0; uint32_t k4 = 0;
switch (len & 15) { switch (len & 15) {
case 15: k4 ^= tail[14] << 16; case 15:
case 14: k4 ^= tail[13] << 8; k4 ^= tail[14] << 16;
case 13: k4 ^= tail[12] << 0; case 14:
k4 *= c4; k4 = ROTL32(k4,18); k4 *= c1; h4 ^= k4; k4 ^= tail[13] << 8;
case 12: k3 ^= tail[11] << 24; case 13:
case 11: k3 ^= tail[10] << 16; k4 ^= tail[12] << 0;
case 10: k3 ^= tail[ 9] << 8; k4 *= c4;
case 9: k3 ^= tail[ 8] << 0; k4 = ROTL32(k4, 18);
k3 *= c3; k3 = ROTL32(k3,17); k3 *= c4; h3 ^= k3; k4 *= c1;
case 8: k2 ^= tail[ 7] << 24; h4 ^= k4;
case 7: k2 ^= tail[ 6] << 16; case 12:
case 6: k2 ^= tail[ 5] << 8; k3 ^= tail[11] << 24;
case 5: k2 ^= tail[ 4] << 0; case 11:
k2 *= c2; k2 = ROTL32(k2,16); k2 *= c3; h2 ^= k2; k3 ^= tail[10] << 16;
case 4: k1 ^= tail[ 3] << 24; case 10:
case 3: k1 ^= tail[ 2] << 16; k3 ^= tail[9] << 8;
case 2: k1 ^= tail[ 1] << 8; case 9:
case 1: k1 ^= tail[ 0] << 0; k3 ^= tail[8] << 0;
k1 *= c1; k1 = ROTL32(k1,15); k1 *= c2; h1 ^= k1; k3 *= c3;
k3 = ROTL32(k3, 17);
k3 *= c4;
h3 ^= k3;
case 8:
k2 ^= tail[7] << 24;
case 7:
k2 ^= tail[6] << 16;
case 6:
k2 ^= tail[5] << 8;
case 5:
k2 ^= tail[4] << 0;
k2 *= c2;
k2 = ROTL32(k2, 16);
k2 *= c3;
h2 ^= k2;
case 4:
k1 ^= tail[3] << 24;
case 3:
k1 ^= tail[2] << 16;
case 2:
k1 ^= tail[1] << 8;
case 1:
k1 ^= tail[0] << 0;
k1 *= c1;
k1 = ROTL32(k1, 15);
k1 *= c2;
h1 ^= k1;
}; };
h1 ^= len; h2 ^= len; h3 ^= len; h4 ^= len; h1 ^= len;
h1 += h2; h1 += h3; h1 += h4; h2 ^= len;
h2 += h1; h3 += h1; h4 += h1; h3 ^= len;
FMIX32(h1); FMIX32(h2); FMIX32(h3); FMIX32(h4); h4 ^= len;
h1 += h2; h1 += h3; h1 += h4; h1 += h2;
h2 += h1; h3 += h1; h4 += h1; h1 += h3;
h1 += h4;
h2 += h1;
h3 += h1;
h4 += h1;
FMIX32(h1);
FMIX32(h2);
FMIX32(h3);
FMIX32(h4);
h1 += h2;
h1 += h3;
h1 += h4;
h2 += h1;
h3 += h1;
h4 += h1;
((uint32_t *)out)[0] = h1; ((uint32_t *)out)[0] = h1;
((uint32_t *)out)[1] = h2; ((uint32_t *)out)[1] = h2;
((uint32_t *)out)[2] = h3; ((uint32_t *)out)[2] = h3;
@ -597,15 +680,13 @@ static void MM86128(const void *key, const int len, uint32_t seed, void *out) {
} }
// hashmap_sip returns a hash value for `data` using SipHash-2-4. // hashmap_sip returns a hash value for `data` using SipHash-2-4.
uint64_t hashmap_sip(const void *data, size_t len, uint64_t hashmap_sip(const void *data, size_t len, uint64_t seed0, uint64_t seed1)
uint64_t seed0, uint64_t seed1)
{ {
return SIP64((uint8_t *)data, len, seed0, seed1); return SIP64((uint8_t *)data, len, seed0, seed1);
} }
// hashmap_murmur returns a hash value for `data` using Murmur3_86_128. // hashmap_murmur returns a hash value for `data` using Murmur3_86_128.
uint64_t hashmap_murmur(const void *data, size_t len, uint64_t hashmap_murmur(const void *data, size_t len, uint64_t seed0, uint64_t seed1)
uint64_t seed0, uint64_t seed1)
{ {
char out[16]; char out[16];
MM86128(data, len, seed0, &out); MM86128(data, len, seed0, &out);
@ -619,7 +700,8 @@ uint64_t hashmap_murmur(const void *data, size_t len,
//============================================================================== //==============================================================================
#ifdef HASHMAP_TEST #ifdef HASHMAP_TEST
static size_t deepcount(struct hashmap *map) { static size_t deepcount(struct hashmap *map)
{
size_t count = 0; size_t count = 0;
for (size_t i = 0; i < map->nbuckets; i++) { for (size_t i = 0; i < map->nbuckets; i++) {
if (bucket_at(map, i)->dib) { if (bucket_at(map, i)->dib) {
@ -629,23 +711,23 @@ static size_t deepcount(struct hashmap *map) {
return count; return count;
} }
# pragma GCC diagnostic ignored "-Wextra" # pragma GCC diagnostic ignored "-Wextra"
# include "core/hashmap.h"
# include <assert.h>
# include <stdio.h>
# include <stdlib.h> # include <stdlib.h>
# include <string.h> # include <string.h>
# include <time.h> # include <time.h>
#include <assert.h>
#include <stdio.h>
#include "hashmap.h"
static bool rand_alloc_fail = false; static bool rand_alloc_fail = false;
static int rand_alloc_fail_odds = 3; // 1 in 3 chance malloc will fail. static int rand_alloc_fail_odds = 3; // 1 in 3 chance malloc will fail.
static uintptr_t total_allocs = 0; static uintptr_t total_allocs = 0;
static uintptr_t total_mem = 0; static uintptr_t total_mem = 0;
static void *xmalloc(size_t size) { static void *xmalloc(size_t size)
{
if (rand_alloc_fail && rand() % rand_alloc_fail_odds == 0) { if (rand_alloc_fail && rand() % rand_alloc_fail_odds == 0) {
return NULL; return NULL;
} }
@ -657,7 +739,8 @@ static void *xmalloc(size_t size) {
return (char *)mem + sizeof(uintptr_t); return (char *)mem + sizeof(uintptr_t);
} }
static void xfree(void *ptr) { static void xfree(void *ptr)
{
if (ptr) { if (ptr) {
total_mem -= *(uintptr_t *)((char *)ptr - sizeof(uintptr_t)); total_mem -= *(uintptr_t *)((char *)ptr - sizeof(uintptr_t));
free((char *)ptr - sizeof(uintptr_t)); free((char *)ptr - sizeof(uintptr_t));
@ -665,7 +748,8 @@ static void xfree(void *ptr) {
} }
} }
static void shuffle(void *array, size_t numels, size_t elsize) { static void shuffle(void *array, size_t numels, size_t elsize)
{
char tmp[elsize]; char tmp[elsize];
char *arr = array; char *arr = array;
for (size_t i = 0; i < numels - 1; i++) { for (size_t i = 0; i < numels - 1; i++) {
@ -676,37 +760,45 @@ static void shuffle(void *array, size_t numels, size_t elsize) {
} }
} }
static bool iter_ints(const void *item, void *udata) { static bool iter_ints(const void *item, void *udata)
{
int *vals = *(int **)udata; int *vals = *(int **)udata;
vals[*(int *)item] = 1; vals[*(int *)item] = 1;
return true; return true;
} }
static int compare_ints(const void *a, const void *b) { static int compare_ints(const void *a, const void *b)
{
return *(int *)a - *(int *)b; return *(int *)a - *(int *)b;
} }
static int compare_ints_udata(const void *a, const void *b, void *udata) { static int compare_ints_udata(const void *a, const void *b, void *udata)
{
return *(int *)a - *(int *)b; return *(int *)a - *(int *)b;
} }
static int compare_strs(const void *a, const void *b, void *udata) { static int compare_strs(const void *a, const void *b, void *udata)
{
return strcmp(*(char **)a, *(char **)b); return strcmp(*(char **)a, *(char **)b);
} }
static uint64_t hash_int(const void *item, uint64_t seed0, uint64_t seed1) { static uint64_t hash_int(const void *item, uint64_t seed0, uint64_t seed1)
{
return hashmap_murmur(item, sizeof(int), seed0, seed1); return hashmap_murmur(item, sizeof(int), seed0, seed1);
} }
static uint64_t hash_str(const void *item, uint64_t seed0, uint64_t seed1) { static uint64_t hash_str(const void *item, uint64_t seed0, uint64_t seed1)
{
return hashmap_murmur(*(char **)item, strlen(*(char **)item), seed0, seed1); return hashmap_murmur(*(char **)item, strlen(*(char **)item), seed0, seed1);
} }
static void free_str(void *item) { static void free_str(void *item)
{
xfree(*(char **)item); xfree(*(char **)item);
} }
static void all() { static void all()
{
int seed = getenv("SEED") ? atoi(getenv("SEED")) : time(NULL); int seed = getenv("SEED") ? atoi(getenv("SEED")) : time(NULL);
int N = getenv("N") ? atoi(getenv("N")) : 2000; int N = getenv("N") ? atoi(getenv("N")) : 2000;
printf("seed=%d, count=%d, item_size=%zu\n", seed, N, sizeof(int)); printf("seed=%d, count=%d, item_size=%zu\n", seed, N, sizeof(int));
@ -719,15 +811,17 @@ static void all() {
assert(hashmap_murmur("hello", 5, 1, 2) == 1682575153221130884); assert(hashmap_murmur("hello", 5, 1, 2) == 1682575153221130884);
int *vals; int *vals;
while (!(vals = xmalloc(N * sizeof(int)))) {} while (!(vals = xmalloc(N * sizeof(int)))) {
}
for (int i = 0; i < N; i++) { for (int i = 0; i < N; i++) {
vals[i] = i; vals[i] = i;
} }
struct hashmap *map; struct hashmap *map;
while (!(map = hashmap_new(sizeof(int), 0, seed, seed, while (!(
hash_int, compare_ints_udata, NULL, NULL))) {} map = hashmap_new(sizeof(int), 0, seed, seed, hash_int, compare_ints_udata, NULL, NULL))) {
}
shuffle(vals, N, sizeof(int)); shuffle(vals, N, sizeof(int));
for (int i = 0; i < N; i++) { for (int i = 0; i < N; i++) {
// // printf("== %d ==\n", vals[i]); // // printf("== %d ==\n", vals[i]);
@ -772,7 +866,8 @@ static void all() {
} }
int *vals2; int *vals2;
while (!(vals2 = xmalloc(N * sizeof(int)))) {} while (!(vals2 = xmalloc(N * sizeof(int)))) {
}
memset(vals2, 0, N * sizeof(int)); memset(vals2, 0, N * sizeof(int));
assert(hashmap_scan(map, iter_ints, &vals2)); assert(hashmap_scan(map, iter_ints, &vals2));
@ -817,7 +912,6 @@ static void all() {
assert(prev_cap < map->cap); assert(prev_cap < map->cap);
assert(map->count == 0); assert(map->count == 0);
for (int i = 0; i < N; i++) { for (int i = 0; i < N; i++) {
while (true) { while (true) {
assert(!hashmap_set(map, &vals[i])); assert(!hashmap_set(map, &vals[i]));
@ -835,15 +929,17 @@ static void all() {
xfree(vals); xfree(vals);
while (
while (!(map = hashmap_new(sizeof(char*), 0, seed, seed, !(map = hashmap_new(sizeof(char *), 0, seed, seed, hash_str, compare_strs, free_str, NULL)))
hash_str, compare_strs, free_str, NULL))); ;
for (int i = 0; i < N; i++) { for (int i = 0; i < N; i++) {
char *str; char *str;
while (!(str = xmalloc(16))); while (!(str = xmalloc(16)))
;
sprintf(str, "s%i", i); sprintf(str, "s%i", i);
while(!hashmap_set(map, &str)); while (!hashmap_set(map, &str))
;
} }
hashmap_clear(map, false); hashmap_clear(map, false);
@ -851,9 +947,11 @@ static void all() {
for (int i = 0; i < N; i++) { for (int i = 0; i < N; i++) {
char *str; char *str;
while (!(str = xmalloc(16))); while (!(str = xmalloc(16)))
;
sprintf(str, "s%i", i); sprintf(str, "s%i", i);
while(!hashmap_set(map, &str)); while (!hashmap_set(map, &str))
;
} }
hashmap_free(map); hashmap_free(map);
@ -864,7 +962,9 @@ static void all() {
} }
} }
#define bench(name, N, code) {{ \ # define bench(name, N, code) \
{ \
{ \
if (strlen(name) > 0) { \ if (strlen(name) > 0) { \
printf("%-14s ", name); \ printf("%-14s ", name); \
} \ } \
@ -878,11 +978,8 @@ static void all() {
clock_t end = clock(); \ clock_t end = clock(); \
double elapsed_secs = (double)(end - begin) / CLOCKS_PER_SEC; \ double elapsed_secs = (double)(end - begin) / CLOCKS_PER_SEC; \
double bytes_sec = (double)bytes / elapsed_secs; \ double bytes_sec = (double)bytes / elapsed_secs; \
printf("%d ops in %.3f secs, %.0f ns/op, %.0f op/sec", \ printf("%d ops in %.3f secs, %.0f ns/op, %.0f op/sec", N, elapsed_secs, \
N, elapsed_secs, \ elapsed_secs / (double)N * 1e9, (double)N / elapsed_secs); \
elapsed_secs/(double)N*1e9, \
(double)N/elapsed_secs \
); \
if (bytes > 0) { \ if (bytes > 0) { \
printf(", %.1f GB/sec", bytes_sec / 1024 / 1024 / 1024); \ printf(", %.1f GB/sec", bytes_sec / 1024 / 1024 / 1024); \
} \ } \
@ -895,15 +992,16 @@ static void all() {
printf(", %.2f allocs/op", (double)used_allocs / N); \ printf(", %.2f allocs/op", (double)used_allocs / N); \
} \ } \
printf("\n"); \ printf("\n"); \
}} } \
}
static void benchmarks() { static void benchmarks()
{
int seed = getenv("SEED") ? atoi(getenv("SEED")) : time(NULL); int seed = getenv("SEED") ? atoi(getenv("SEED")) : time(NULL);
int N = getenv("N") ? atoi(getenv("N")) : 5000000; int N = getenv("N") ? atoi(getenv("N")) : 5000000;
printf("seed=%d, count=%d, item_size=%zu\n", seed, N, sizeof(int)); printf("seed=%d, count=%d, item_size=%zu\n", seed, N, sizeof(int));
srand(seed); srand(seed);
int *vals = xmalloc(N * sizeof(int)); int *vals = xmalloc(N * sizeof(int));
for (int i = 0; i < N; i++) { for (int i = 0; i < N; i++) {
vals[i] = i; vals[i] = i;
@ -914,44 +1012,37 @@ static void benchmarks() {
struct hashmap *map; struct hashmap *map;
shuffle(vals, N, sizeof(int)); shuffle(vals, N, sizeof(int));
map = hashmap_new(sizeof(int), 0, seed, seed, hash_int, compare_ints_udata, map = hashmap_new(sizeof(int), 0, seed, seed, hash_int, compare_ints_udata, NULL, NULL);
NULL, NULL);
bench("set", N, { bench("set", N, {
int *v = hashmap_set(map, &vals[i]); int *v = hashmap_set(map, &vals[i]);
assert(!v); assert(!v);
}) }) shuffle(vals, N, sizeof(int));
shuffle(vals, N, sizeof(int));
bench("get", N, { bench("get", N, {
int *v = hashmap_get(map, &vals[i]); int *v = hashmap_get(map, &vals[i]);
assert(v && *v == vals[i]); assert(v && *v == vals[i]);
}) }) shuffle(vals, N, sizeof(int));
shuffle(vals, N, sizeof(int));
bench("delete", N, { bench("delete", N, {
int *v = hashmap_delete(map, &vals[i]); int *v = hashmap_delete(map, &vals[i]);
assert(v && *v == vals[i]); assert(v && *v == vals[i]);
}) }) hashmap_free(map);
hashmap_free(map);
map = hashmap_new(sizeof(int), N, seed, seed, hash_int, compare_ints_udata, map = hashmap_new(sizeof(int), N, seed, seed, hash_int, compare_ints_udata, NULL, NULL);
NULL, NULL);
bench("set (cap)", N, { bench("set (cap)", N, {
int *v = hashmap_set(map, &vals[i]); int *v = hashmap_set(map, &vals[i]);
assert(!v); assert(!v);
}) }) shuffle(vals, N, sizeof(int));
shuffle(vals, N, sizeof(int));
bench("get (cap)", N, { bench("get (cap)", N, {
int *v = hashmap_get(map, &vals[i]); int *v = hashmap_get(map, &vals[i]);
assert(v && *v == vals[i]); assert(v && *v == vals[i]);
}) }) shuffle(vals, N, sizeof(int));
shuffle(vals, N, sizeof(int)); bench("delete (cap)", N,
bench("delete (cap)" , N, { {
int *v = hashmap_delete(map, &vals[i]); int *v = hashmap_delete(map, &vals[i]);
assert(v && *v == vals[i]); assert(v && *v == vals[i]);
}) })
hashmap_free(map); hashmap_free(map);
xfree(vals); xfree(vals);
if (total_allocs != 0) { if (total_allocs != 0) {
@ -960,7 +1051,8 @@ static void benchmarks() {
} }
} }
int main() { int main()
{
hashmap_set_allocator(xmalloc, xfree); hashmap_set_allocator(xmalloc, xfree);
if (getenv("BENCH")) { if (getenv("BENCH")) {
@ -973,8 +1065,4 @@ int main() {
} }
} }
#endif #endif

View File

@ -15,11 +15,11 @@ https://github.com/benhoyt/inih
# define _CRT_SECURE_NO_WARNINGS # define _CRT_SECURE_NO_WARNINGS
#endif #endif
#include <stdio.h> #include "core/ini.h"
#include <ctype.h>
#include <string.h>
#include "ini.h" #include <ctype.h>
#include <stdio.h>
#include <string.h>
#if !INI_USE_STACK #if !INI_USE_STACK
# if INI_CUSTOM_ALLOCATOR # if INI_CUSTOM_ALLOCATOR
@ -39,7 +39,8 @@ void* ini_realloc(void* ptr, size_t size);
#define MAX_NAME 50 #define MAX_NAME 50
/* Used by ini_parse_string() to keep track of string parsing state. */ /* Used by ini_parse_string() to keep track of string parsing state. */
typedef struct { typedef struct
{
const char *ptr; const char *ptr;
size_t num_left; size_t num_left;
} ini_parse_string_ctx; } ini_parse_string_ctx;
@ -94,8 +95,7 @@ static char* strncpy0(char* dest, const char* src, size_t size)
} }
/* See documentation in header file. */ /* See documentation in header file. */
int ini_parse_stream(ini_reader reader, void* stream, ini_handler handler, int ini_parse_stream(ini_reader reader, void *stream, ini_handler handler, void *user)
void* user)
{ {
/* Uses a fair bit of stack (use heap instead if you need to) */ /* Uses a fair bit of stack (use heap instead if you need to) */
#if INI_USE_STACK #if INI_USE_STACK
@ -158,8 +158,7 @@ int ini_parse_stream(ini_reader reader, void* stream, ini_handler handler,
start = line; start = line;
#if INI_ALLOW_BOM #if INI_ALLOW_BOM
if (lineno == 1 && (unsigned char)start[0] == 0xEF && if (lineno == 1 && (unsigned char)start[0] == 0xEF && (unsigned char)start[1] == 0xBB &&
(unsigned char)start[1] == 0xBB &&
(unsigned char)start[2] == 0xBF) { (unsigned char)start[2] == 0xBF) {
start += 3; start += 3;
} }
@ -188,13 +187,11 @@ int ini_parse_stream(ini_reader reader, void* stream, ini_handler handler,
if (!HANDLER(user, section, NULL, NULL) && !error) if (!HANDLER(user, section, NULL, NULL) && !error)
error = lineno; error = lineno;
#endif #endif
} } else if (!error) {
else if (!error) {
/* No ']' found on section line */ /* No ']' found on section line */
error = lineno; error = lineno;
} }
} } else if (*start) {
else if (*start) {
/* Not a comment, must be a name[=:]value pair */ /* Not a comment, must be a name[=:]value pair */
end = find_chars_or_comment(start, "=:"); end = find_chars_or_comment(start, "=:");
if (*end == '=' || *end == ':') { if (*end == '=' || *end == ':') {
@ -213,8 +210,7 @@ int ini_parse_stream(ini_reader reader, void* stream, ini_handler handler,
strncpy0(prev_name, name, sizeof(prev_name)); strncpy0(prev_name, name, sizeof(prev_name));
if (!HANDLER(user, section, name, value) && !error) if (!HANDLER(user, section, name, value) && !error)
error = lineno; error = lineno;
} } else if (!error) {
else if (!error) {
/* No '=' or ':' found on name[=:]value line */ /* No '=' or ':' found on name[=:]value line */
#if INI_ALLOW_NO_VALUE #if INI_ALLOW_NO_VALUE
*end = '\0'; *end = '\0';
@ -262,7 +258,8 @@ int ini_parse(const char* filename, ini_handler handler, void* user)
/* An ini_reader function to read the next line from a string buffer. This /* An ini_reader function to read the next line from a string buffer. This
is the fgets() equivalent used by ini_parse_string(). */ is the fgets() equivalent used by ini_parse_string(). */
static char* ini_reader_string(char* str, int num, void* stream) { static char *ini_reader_string(char *str, int num, void *stream)
{
ini_parse_string_ctx *ctx = (ini_parse_string_ctx *)stream; ini_parse_string_ctx *ctx = (ini_parse_string_ctx *)stream;
const char *ctx_ptr = ctx->ptr; const char *ctx_ptr = ctx->ptr;
size_t ctx_num_left = ctx->num_left; size_t ctx_num_left = ctx->num_left;
@ -288,11 +285,11 @@ static char* ini_reader_string(char* str, int num, void* stream) {
} }
/* See documentation in header file. */ /* See documentation in header file. */
int ini_parse_string(const char* string, ini_handler handler, void* user) { int ini_parse_string(const char *string, ini_handler handler, void *user)
{
ini_parse_string_ctx ctx; ini_parse_string_ctx ctx;
ctx.ptr = string; ctx.ptr = string;
ctx.num_left = strlen(string); ctx.num_left = strlen(string);
return ini_parse_stream((ini_reader)ini_reader_string, &ctx, handler, return ini_parse_stream((ini_reader)ini_reader_string, &ctx, handler, user);
user);
} }

4
lib/src/core/lerror.c Normal file
View File

@ -0,0 +1,4 @@
#include "core/lerror.h"
jmp_buf eLaika_errStack[LAIKA_MAXERRORS];
int eLaika_errIndx = -1;

View File

@ -1,6 +1,6 @@
#include "lmem.h" #include "core/lmem.h"
#include "lerror.h" #include "core/lerror.h"
void *laikaM_realloc(void *buf, size_t sz) void *laikaM_realloc(void *buf, size_t sz)
{ {

View File

@ -1,4 +1,4 @@
#include "lsodium.h" #include "core/lsodium.h"
#include <string.h> #include <string.h>

View File

@ -1,6 +1,6 @@
#include "ltask.h" #include "core/ltask.h"
#include "lmem.h" #include "core/lmem.h"
/* this is the only reason C11 support is needed, i cba to write windows/linux specific stuff to get /* this is the only reason C11 support is needed, i cba to write windows/linux specific stuff to get
the current time in ms also side note: microsoft? more like micropenis */ the current time in ms also side note: microsoft? more like micropenis */

1
lib/src/core/lvm.c Normal file
View File

@ -0,0 +1 @@
#include "core/lvm.h"

View File

@ -1,4 +0,0 @@
#include "lerror.h"
jmp_buf eLaika_errStack[LAIKA_MAXERRORS];
int eLaika_errIndx = -1;

View File

@ -1 +0,0 @@
#include "lvm.h"

View File

@ -1,4 +1,4 @@
#include "lpacket.h" #include "net/lpacket.h"
#ifdef DEBUG #ifdef DEBUG
const char *laikaD_getPacketName(LAIKAPKT_ID id) const char *laikaD_getPacketName(LAIKAPKT_ID id)

View File

@ -1,7 +1,7 @@
#include "lpeer.h" #include "net/lpeer.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
struct sLaika_peer *laikaS_newPeer(struct sLaika_peerPacketInfo *pktTbl, struct sLaika_peer *laikaS_newPeer(struct sLaika_peerPacketInfo *pktTbl,
struct sLaika_pollList *pList, pollFailEvent onPollFail, struct sLaika_pollList *pList, pollFailEvent onPollFail,
@ -71,7 +71,7 @@ void laikaS_startOutPacket(struct sLaika_peer *peer, LAIKAPKT_ID id)
laikaS_writeByte(sock, id); laikaS_writeByte(sock, id);
peer->outStart = sock->outCount; peer->outStart = laikaM_countVector(sock->outBuf);
if (peer->useSecure) { /* if we're encrypting this packet, append the nonce right after the if (peer->useSecure) { /* if we're encrypting this packet, append the nonce right after the
packet ID */ packet ID */
uint8_t nonce[crypto_secretbox_NONCEBYTES]; uint8_t nonce[crypto_secretbox_NONCEBYTES];
@ -88,26 +88,26 @@ int laikaS_endOutPacket(struct sLaika_peer *peer)
if (peer->useSecure) { if (peer->useSecure) {
/* make sure we have enough space */ /* make sure we have enough space */
laikaM_growarray(uint8_t, sock->outBuf, crypto_secretbox_MACBYTES, sock->outCount, laikaM_growVector(uint8_t, sock->outBuf, crypto_secretbox_MACBYTES);
sock->outCap);
/* packet body starts after the id & nonce */ /* packet body starts after the id & nonce */
body = &sock->outBuf[peer->outStart + crypto_secretbox_NONCEBYTES]; body = &sock->outBuf[peer->outStart + crypto_secretbox_NONCEBYTES];
/* encrypt packet body in-place */ /* encrypt packet body in-place */
if (crypto_secretbox_easy(body, body, if (crypto_secretbox_easy(body, body,
(sock->outCount - peer->outStart) - crypto_secretbox_NONCEBYTES, (laikaM_countVector(sock->outBuf) - peer->outStart) -
crypto_secretbox_NONCEBYTES,
&sock->outBuf[peer->outStart], peer->outKey) != 0) { &sock->outBuf[peer->outStart], peer->outKey) != 0) {
LAIKA_ERROR("Failed to encrypt packet!\n"); LAIKA_ERROR("Failed to encrypt packet!\n");
} }
sock->outCount += crypto_secretbox_MACBYTES; laikaM_countVector(sock->outBuf) += crypto_secretbox_MACBYTES;
} }
/* add to pollList's out queue */ /* add to pollList's out queue */
laikaP_pushOutQueue(peer->pList, &peer->sock); laikaP_pushOutQueue(peer->pList, &peer->sock);
/* return packet size and prepare for next outPacket */ /* return packet size and prepare for next outPacket */
sz = sock->outCount - peer->outStart; sz = laikaM_countVector(sock->outBuf) - peer->outStart;
peer->outStart = -1; peer->outStart = -1;
return sz; return sz;
} }
@ -148,30 +148,31 @@ void laikaS_startInPacket(struct sLaika_peer *peer, bool variadic)
if (peer->useSecure && !variadic && peer->pktSize != 0) if (peer->useSecure && !variadic && peer->pktSize != 0)
peer->pktSize += crypto_secretbox_MACBYTES + crypto_secretbox_NONCEBYTES; peer->pktSize += crypto_secretbox_MACBYTES + crypto_secretbox_NONCEBYTES;
peer->inStart = sock->inCount; peer->inStart = laikaM_countVector(sock->inBuf);
} }
int laikaS_endInPacket(struct sLaika_peer *peer) int laikaS_endInPacket(struct sLaika_peer *peer)
{ {
struct sLaika_socket *sock = &peer->sock; struct sLaika_socket *sock = &peer->sock;
uint8_t *body; uint8_t *body;
size_t sz = sock->inCount - peer->inStart; size_t sz = laikaM_countVector(sock->inBuf) - peer->inStart;
if (peer->useSecure && sz > crypto_secretbox_MACBYTES + crypto_secretbox_NONCEBYTES) { if (peer->useSecure && sz > crypto_secretbox_MACBYTES + crypto_secretbox_NONCEBYTES) {
body = &sock->inBuf[peer->inStart + crypto_secretbox_NONCEBYTES]; body = &sock->inBuf[peer->inStart + crypto_secretbox_NONCEBYTES];
/* decrypt packet body in-place */ /* decrypt packet body in-place */
if (crypto_secretbox_open_easy( if (crypto_secretbox_open_easy(body, body,
body, body, (sock->inCount - peer->inStart) - crypto_secretbox_NONCEBYTES, (laikaM_countVector(sock->inBuf) - peer->inStart) -
crypto_secretbox_NONCEBYTES,
&sock->inBuf[peer->inStart], peer->inKey) != 0) { &sock->inBuf[peer->inStart], peer->inKey) != 0) {
LAIKA_ERROR("Failed to decrypt packet!\n"); LAIKA_ERROR("Failed to decrypt packet!\n");
} }
/* decrypted message is smaller now */ /* decrypted message is smaller now */
sock->inCount -= crypto_secretbox_MACBYTES; laikaM_countVector(sock->inBuf) -= crypto_secretbox_MACBYTES;
/* remove nonce */ /* remove nonce */
laikaM_rmvarray(sock->inBuf, sock->inCount, peer->inStart, crypto_secretbox_NONCEBYTES); laikaM_rmvVector(sock->inBuf, peer->inStart, crypto_secretbox_NONCEBYTES);
sz -= crypto_secretbox_MACBYTES + crypto_secretbox_NONCEBYTES; sz -= crypto_secretbox_MACBYTES + crypto_secretbox_NONCEBYTES;
} }
@ -254,18 +255,19 @@ bool laikaS_handlePeerIn(struct sLaika_socket *sock)
default: default:
_HandlePacketBody: _HandlePacketBody:
/* try grabbing the rest of the packet */ /* try grabbing the rest of the packet */
if (laikaS_rawRecv(&peer->sock, peer->pktSize - peer->sock.inCount, &recvd) != RAWSOCK_OK) if (laikaS_rawRecv(&peer->sock, peer->pktSize - laikaM_countVector(peer->sock.inBuf),
&recvd) != RAWSOCK_OK)
return false; return false;
/* have we received the full packet? */ /* have we received the full packet? */
if (peer->pktSize == peer->sock.inCount) { if (peer->pktSize == laikaM_countVector(peer->sock.inBuf)) {
peer->pktSize = laikaS_endInPacket(peer); peer->pktSize = laikaS_endInPacket(peer);
/* dispatch to packet handler */ /* dispatch to packet handler */
peer->packetTbl[peer->pktID].handler(peer, peer->pktSize, peer->uData); peer->packetTbl[peer->pktID].handler(peer, peer->pktSize, peer->uData);
/* reset */ /* reset */
peer->sock.inCount = 0; laikaM_countVector(peer->sock.inBuf) = 0;
peer->pktID = LAIKAPKT_MAXNONE; peer->pktID = LAIKAPKT_MAXNONE;
} }
@ -280,10 +282,10 @@ bool laikaS_handlePeerOut(struct sLaika_socket *sock)
struct sLaika_peer *peer = (struct sLaika_peer *)sock; struct sLaika_peer *peer = (struct sLaika_peer *)sock;
int sent; int sent;
if (peer->sock.outCount == 0) /* sanity check */ if (laikaM_countVector(peer->sock.outBuf) == 0) /* sanity check */
return true; return true;
switch (laikaS_rawSend(&peer->sock, peer->sock.outCount, &sent)) { switch (laikaS_rawSend(&peer->sock, laikaM_countVector(peer->sock.outBuf), &sent)) {
case RAWSOCK_OK: /* we're ok! */ case RAWSOCK_OK: /* we're ok! */
/* if POLLOUT was set, unset it */ /* if POLLOUT was set, unset it */
laikaP_rmvPollOut(peer->pList, &peer->sock); laikaP_rmvPollOut(peer->pList, &peer->sock);

View File

@ -1,7 +1,7 @@
#include "lpolllist.h" #include "net/lpolllist.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
/* ===================================[[ Helper Functions ]]==================================== */ /* ===================================[[ Helper Functions ]]==================================== */
@ -34,12 +34,10 @@ void laikaP_initPList(struct sLaika_pollList *pList)
/* setup hashmap */ /* setup hashmap */
pList->sockets = hashmap_new(sizeof(tLaika_hashMapElem), POLLSTARTCAP, 0, 0, elem_hash, pList->sockets = hashmap_new(sizeof(tLaika_hashMapElem), POLLSTARTCAP, 0, 0, elem_hash,
elem_compare, NULL, NULL); elem_compare, NULL, NULL);
pList->revents = NULL; /* laikaP_pollList() will allocate the buffer */
pList->reventCap = POLLSTARTCAP / GROW_FACTOR; /* laikaP_pollList() will allocate these buffer */
pList->reventCount = 0; laikaM_initVector(pList->revents, POLLSTARTCAP / GROW_FACTOR);
pList->outQueue = NULL; laikaM_initVector(pList->outQueue, POLLSTARTCAP / GROW_FACTOR);
pList->outCap = POLLSTARTCAP / GROW_FACTOR;
pList->outCount = 0;
#ifdef LAIKA_USE_EPOLL #ifdef LAIKA_USE_EPOLL
/* setup our epoll */ /* setup our epoll */
@ -48,11 +46,8 @@ void laikaP_initPList(struct sLaika_pollList *pList)
LAIKA_ERROR("epoll_create() failed!\n"); LAIKA_ERROR("epoll_create() failed!\n");
#else #else
pList->fds = NULL; /* laikaP_addSock will allocate the buffer */ /* laikaP_addSock will allocate this buffer */
pList->fdCapacity = laikaM_initVector(pList->fds, POLLSTARTCAP / GROW_FACTOR);
POLLSTARTCAP /
GROW_FACTOR; /* div by GROW_FACTOR since laikaM_growarray multiplies by GROW_FACTOR */
pList->fdCount = 0;
#endif #endif
} }
@ -85,8 +80,8 @@ void laikaP_addSock(struct sLaika_pollList *pList, struct sLaika_socket *sock)
#else #else
/* allocate space in array & add PollFD */ /* allocate space in array & add PollFD */
laikaM_growarray(PollFD, pList->fds, 1, pList->fdCount, pList->fdCapacity); laikaM_growVector(PollFD, pList->fds, 1);
pList->fds[pList->fdCount++] = (PollFD){sock->sock, POLLIN}; pList->fds[laikaM_countVector(pList->fds)++] = (PollFD){sock->sock, POLLIN};
#endif #endif
} }
@ -98,9 +93,9 @@ void laikaP_rmvSock(struct sLaika_pollList *pList, struct sLaika_socket *sock)
hashmap_delete(pList->sockets, &(tLaika_hashMapElem){.fd = sock->sock, .sock = sock}); hashmap_delete(pList->sockets, &(tLaika_hashMapElem){.fd = sock->sock, .sock = sock});
/* make sure peer isn't in outQueue */ /* make sure peer isn't in outQueue */
for (i = 0; i < pList->outCount; i++) { for (i = 0; i < laikaM_countVector(pList->outQueue); i++) {
if ((void *)pList->outQueue[i] == (void *)sock) { if ((void *)pList->outQueue[i] == (void *)sock) {
laikaM_rmvarray(pList->outQueue, pList->outCount, i, 1); laikaM_rmvVector(pList->outQueue, i, 1);
} }
} }
@ -114,10 +109,10 @@ void laikaP_rmvSock(struct sLaika_pollList *pList, struct sLaika_socket *sock)
#else #else
/* search fds for socket, remove it and shrink array */ /* search fds for socket, remove it and shrink array */
for (i = 0; i < pList->fdCount; i++) { for (i = 0; i < laikaM_countVector(pList->fds); i++) {
if (pList->fds[i].fd == sock->sock) { if (pList->fds[i].fd == sock->sock) {
/* remove from array */ /* remove from array */
laikaM_rmvarray(pList->fds, pList->fdCount, i, 1); laikaM_rmvVector(pList->fds, i, 1);
break; break;
} }
} }
@ -140,7 +135,7 @@ void laikaP_addPollOut(struct sLaika_pollList *pList, struct sLaika_socket *sock
int i; int i;
/* search fds for socket, add POLLOUT flag */ /* search fds for socket, add POLLOUT flag */
for (i = 0; i < pList->fdCount; i++) { for (i = 0; i < laikaM_countVector(pList->fds); i++) {
if (pList->fds[i].fd == sock->sock) { if (pList->fds[i].fd == sock->sock) {
pList->fds[i].events = POLLIN | POLLOUT; pList->fds[i].events = POLLIN | POLLOUT;
break; break;
@ -167,7 +162,7 @@ void laikaP_rmvPollOut(struct sLaika_pollList *pList, struct sLaika_socket *sock
int i; int i;
/* search fds for socket, remove POLLOUT flag */ /* search fds for socket, remove POLLOUT flag */
for (i = 0; i < pList->fdCount; i++) { for (i = 0; i < laikaM_countVector(pList->fds); i++) {
if (pList->fds[i].fd == sock->sock) { if (pList->fds[i].fd == sock->sock) {
pList->fds[i].events = POLLIN; pList->fds[i].events = POLLIN;
break; break;
@ -183,18 +178,18 @@ void laikaP_pushOutQueue(struct sLaika_pollList *pList, struct sLaika_socket *so
int i; int i;
/* first, check that we don't have this peer in the queue already */ /* first, check that we don't have this peer in the queue already */
for (i = 0; i < pList->outCount; i++) { for (i = 0; i < laikaM_countVector(pList->outQueue); i++) {
if (pList->outQueue[i] == sock) if (pList->outQueue[i] == sock)
return; /* found it :) */ return; /* found it :) */
} }
laikaM_growarray(struct sLaika_socket *, pList->outQueue, 1, pList->outCount, pList->outCap); laikaM_growVector(struct sLaika_socket *, pList->outQueue, 1);
pList->outQueue[pList->outCount++] = sock; pList->outQueue[laikaM_countVector(pList->outQueue)++] = sock;
} }
void laikaP_resetOutQueue(struct sLaika_pollList *pList) void laikaP_resetOutQueue(struct sLaika_pollList *pList)
{ {
pList->outCount = 0; /* ez lol */ laikaM_countVector(pList->outQueue) = 0; /* ez lol */
} }
void laikaP_flushOutQueue(struct sLaika_pollList *pList) void laikaP_flushOutQueue(struct sLaika_pollList *pList)
@ -203,7 +198,7 @@ void laikaP_flushOutQueue(struct sLaika_pollList *pList)
int i; int i;
/* flush pList's outQueue */ /* flush pList's outQueue */
for (i = 0; i < pList->outCount; i++) { for (i = 0; i < laikaM_countVector(pList->outQueue); i++) {
sock = pList->outQueue[i]; sock = pList->outQueue[i];
LAIKA_DEBUG("sending OUT to %p\n", sock); LAIKA_DEBUG("sending OUT to %p\n", sock);
if (sock->onPollOut && !sock->onPollOut(sock) && sock->onPollFail) if (sock->onPollOut && !sock->onPollOut(sock) && sock->onPollFail)
@ -216,7 +211,7 @@ struct sLaika_pollEvent *laikaP_poll(struct sLaika_pollList *pList, int timeout,
{ {
int nEvents, i; int nEvents, i;
pList->reventCount = 0; /* reset revent array */ laikaM_countVector(pList->revents) = 0; /* reset revent array */
#ifdef LAIKA_USE_EPOLL #ifdef LAIKA_USE_EPOLL
/* fastpath: we store the sLaika_socket* pointer directly in the epoll_data_t, saving us a /* fastpath: we store the sLaika_socket* pointer directly in the epoll_data_t, saving us a
lookup into our socket hashmap not to mention the various improvements epoll() has over lookup into our socket hashmap not to mention the various improvements epoll() has over
@ -229,22 +224,21 @@ struct sLaika_pollEvent *laikaP_poll(struct sLaika_pollList *pList, int timeout,
for (i = 0; i < nEvents; i++) { for (i = 0; i < nEvents; i++) {
/* add event to revent array */ /* add event to revent array */
laikaM_growarray(struct sLaika_pollEvent, pList->revents, 1, pList->reventCount, laikaM_growVector(struct sLaika_pollEvent, pList->revents, 1);
pList->reventCap); pList->revents[laikaM_countVector(pList->revents)++] =
pList->revents[pList->reventCount++] =
(struct sLaika_pollEvent){.sock = pList->ep_events[i].data.ptr, (struct sLaika_pollEvent){.sock = pList->ep_events[i].data.ptr,
.pollIn = pList->ep_events[i].events & EPOLLIN, .pollIn = pList->ep_events[i].events & EPOLLIN,
.pollOut = pList->ep_events[i].events & EPOLLOUT}; .pollOut = pList->ep_events[i].events & EPOLLOUT};
} }
#else #else
nEvents = poll(pList->fds, pList->fdCount, /* poll returns -1 for error, or the number of events */
timeout); /* poll returns -1 for error, or the number of events */ nEvents = poll(pList->fds, laikaM_countVector(pList->fds), timeout);
if (SOCKETERROR(nEvents)) if (SOCKETERROR(nEvents))
LAIKA_ERROR("poll() failed!\n"); LAIKA_ERROR("poll() failed!\n");
/* walk through the returned poll fds, if they have an event, add it to our revents array */ /* walk through the returned poll fds, if they have an event, add it to our revents array */
for (i = 0; i < pList->fdCount && nEvents > 0; i++) { for (i = 0; i < laikaM_countVector(pList->fds) && nEvents > 0; i++) {
PollFD pfd = pList->fds[i]; PollFD pfd = pList->fds[i];
if (pList->fds[i].revents != 0) { if (pList->fds[i].revents != 0) {
/* grab socket from hashmap */ /* grab socket from hashmap */
@ -252,9 +246,8 @@ struct sLaika_pollEvent *laikaP_poll(struct sLaika_pollList *pList, int timeout,
pList->sockets, &(tLaika_hashMapElem){.fd = (SOCKET)pfd.fd}); pList->sockets, &(tLaika_hashMapElem){.fd = (SOCKET)pfd.fd});
/* insert event into revents array */ /* insert event into revents array */
laikaM_growarray(struct sLaika_pollEvent, pList->revents, 1, pList->reventCount, laikaM_growVector(struct sLaika_pollEvent, pList->revents, 1);
pList->reventCap); pList->revents[laikaM_countVector(pList->revents)++] =
pList->revents[pList->reventCount++] =
(struct sLaika_pollEvent){.sock = elem->sock, (struct sLaika_pollEvent){.sock = elem->sock,
.pollIn = pfd.revents & POLLIN, .pollIn = pfd.revents & POLLIN,
.pollOut = pfd.revents & POLLOUT}; .pollOut = pfd.revents & POLLOUT};
@ -264,7 +257,7 @@ struct sLaika_pollEvent *laikaP_poll(struct sLaika_pollList *pList, int timeout,
} }
#endif #endif
*_nevents = pList->reventCount; *_nevents = laikaM_countVector(pList->revents);
/* return revents array */ /* return revents array */
return pList->revents; return pList->revents;

View File

@ -1,10 +1,10 @@
#include "lsocket.h" #include "net/lsocket.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
#include "lpacket.h" #include "core/lsodium.h"
#include "lpolllist.h" #include "net/lpacket.h"
#include "lsodium.h" #include "net/lpolllist.h"
static int _LNSetup = 0; static int _LNSetup = 0;
@ -50,12 +50,8 @@ void laikaS_initSocket(struct sLaika_socket *sock, pollEvent onPollIn, pollEvent
sock->onPollIn = onPollIn; sock->onPollIn = onPollIn;
sock->onPollOut = onPollOut; sock->onPollOut = onPollOut;
sock->uData = uData; sock->uData = uData;
sock->inBuf = NULL; laikaM_initVector(sock->inBuf, 8);
sock->inCap = 8; laikaM_initVector(sock->outBuf, 8);
sock->inCount = 0;
sock->outBuf = NULL;
sock->outCap = 8;
sock->outCount = 0;
sock->flipEndian = false; sock->flipEndian = false;
sock->setPollOut = false; sock->setPollOut = false;
@ -197,44 +193,44 @@ bool laikaS_setNonBlock(struct sLaika_socket *sock)
void laikaS_consumeRead(struct sLaika_socket *sock, size_t sz) void laikaS_consumeRead(struct sLaika_socket *sock, size_t sz)
{ {
laikaM_rmvarray(sock->inBuf, sock->inCount, 0, sz); laikaM_rmvVector(sock->inBuf, 0, sz);
} }
void laikaS_zeroWrite(struct sLaika_socket *sock, size_t sz) void laikaS_zeroWrite(struct sLaika_socket *sock, size_t sz)
{ {
laikaM_growarray(uint8_t, sock->outBuf, sz, sock->outCount, sock->outCap); laikaM_growVector(uint8_t, sock->outBuf, sz);
/* set NULL bytes */ /* set NULL bytes */
memset(&sock->outBuf[sock->outCount], 0, sz); memset(&sock->outBuf[laikaM_countVector(sock->outBuf)], 0, sz);
sock->outCount += sz; laikaM_countVector(sock->outBuf) += sz;
} }
void laikaS_read(struct sLaika_socket *sock, void *buf, size_t sz) void laikaS_read(struct sLaika_socket *sock, void *buf, size_t sz)
{ {
memcpy(buf, sock->inBuf, sz); memcpy(buf, sock->inBuf, sz);
laikaM_rmvarray(sock->inBuf, sock->inCount, 0, sz); laikaM_rmvVector(sock->inBuf, 0, sz);
} }
void laikaS_write(struct sLaika_socket *sock, void *buf, size_t sz) void laikaS_write(struct sLaika_socket *sock, void *buf, size_t sz)
{ {
/* make sure we have enough space to copy the buffer */ /* make sure we have enough space to copy the buffer */
laikaM_growarray(uint8_t, sock->outBuf, sz, sock->outCount, sock->outCap); laikaM_growVector(uint8_t, sock->outBuf, sz);
/* copy the buffer, then increment outCount */ /* copy the buffer, then increment outCount */
memcpy(&sock->outBuf[sock->outCount], buf, sz); memcpy(&sock->outBuf[laikaM_countVector(sock->outBuf)], buf, sz);
sock->outCount += sz; laikaM_countVector(sock->outBuf) += sz;
} }
void laikaS_writeKeyEncrypt(struct sLaika_socket *sock, void *buf, size_t sz, uint8_t *pub) void laikaS_writeKeyEncrypt(struct sLaika_socket *sock, void *buf, size_t sz, uint8_t *pub)
{ {
/* make sure we have enough space to encrypt the buffer */ /* make sure we have enough space to encrypt the buffer */
laikaM_growarray(uint8_t, sock->outBuf, LAIKAENC_SIZE(sz), sock->outCount, sock->outCap); laikaM_growVector(uint8_t, sock->outBuf, LAIKAENC_SIZE(sz));
/* encrypt the buffer into outBuf */ /* encrypt the buffer into outBuf */
if (crypto_box_seal(&sock->outBuf[sock->outCount], buf, sz, pub) != 0) if (crypto_box_seal(&sock->outBuf[laikaM_countVector(sock->outBuf)], buf, sz, pub) != 0)
LAIKA_ERROR("Failed to encrypt!\n"); LAIKA_ERROR("Failed to encrypt!\n");
sock->outCount += LAIKAENC_SIZE(sz); laikaM_countVector(sock->outBuf) += LAIKAENC_SIZE(sz);
} }
void laikaS_readKeyDecrypt(struct sLaika_socket *sock, void *buf, size_t sz, uint8_t *pub, void laikaS_readKeyDecrypt(struct sLaika_socket *sock, void *buf, size_t sz, uint8_t *pub,
@ -244,21 +240,21 @@ void laikaS_readKeyDecrypt(struct sLaika_socket *sock, void *buf, size_t sz, uin
if (crypto_box_seal_open(buf, sock->inBuf, LAIKAENC_SIZE(sz), pub, priv) != 0) if (crypto_box_seal_open(buf, sock->inBuf, LAIKAENC_SIZE(sz), pub, priv) != 0)
LAIKA_ERROR("Failed to decrypt!\n"); LAIKA_ERROR("Failed to decrypt!\n");
laikaM_rmvarray(sock->inBuf, sock->inCount, 0, LAIKAENC_SIZE(sz)); laikaM_rmvVector(sock->inBuf, 0, LAIKAENC_SIZE(sz));
} }
void laikaS_writeByte(struct sLaika_socket *sock, uint8_t data) void laikaS_writeByte(struct sLaika_socket *sock, uint8_t data)
{ {
laikaM_growarray(uint8_t, sock->outBuf, 1, sock->outCount, sock->outCap); laikaM_growVector(uint8_t, sock->outBuf, 1);
sock->outBuf[sock->outCount++] = data; sock->outBuf[laikaM_countVector(sock->outBuf)++] = data;
} }
uint8_t laikaS_readByte(struct sLaika_socket *sock) uint8_t laikaS_readByte(struct sLaika_socket *sock)
{ {
uint8_t tmp = *sock->inBuf; uint8_t tmp = *sock->inBuf;
/* pop 1 byte */ /* consume 1 byte */
laikaM_rmvarray(sock->inBuf, sock->inCount, 0, 1); laikaM_rmvVector(sock->inBuf, 0, 1);
return tmp; return tmp;
} }
@ -302,15 +298,16 @@ void laikaS_writeInt(struct sLaika_socket *sock, void *buf, size_t sz)
RAWSOCKCODE laikaS_rawRecv(struct sLaika_socket *sock, size_t sz, int *processed) RAWSOCKCODE laikaS_rawRecv(struct sLaika_socket *sock, size_t sz, int *processed)
{ {
RAWSOCKCODE errCode = RAWSOCK_OK; RAWSOCKCODE errCode = RAWSOCK_OK;
int i, rcvd, start = sock->inCount; int i, rcvd, start = laikaM_countVector(sock->inBuf);
/* sanity check */ /* sanity check */
if (sz == 0) if (sz == 0)
return RAWSOCK_OK; return RAWSOCK_OK;
/* make sure we have enough space to recv */ /* make sure we have enough space to recv */
laikaM_growarray(uint8_t, sock->inBuf, sz, sock->inCount, sock->inCap); laikaM_growVector(uint8_t, sock->inBuf, sz);
rcvd = recv(sock->sock, (buffer_t *)&sock->inBuf[sock->inCount], sz, LN_MSG_NOSIGNAL); rcvd = recv(sock->sock, (buffer_t *)&sock->inBuf[laikaM_countVector(sock->inBuf)], sz,
LN_MSG_NOSIGNAL);
if (rcvd == 0) { if (rcvd == 0) {
errCode = RAWSOCK_CLOSED; errCode = RAWSOCK_CLOSED;
@ -340,7 +337,7 @@ RAWSOCKCODE laikaS_rawRecv(struct sLaika_socket *sock, size_t sz, int *processed
#endif #endif
/* recv() worked, add rcvd to inCount */ /* recv() worked, add rcvd to inCount */
sock->inCount += rcvd; laikaM_countVector(sock->inBuf) += rcvd;
} }
*processed = rcvd; *processed = rcvd;
return errCode; return errCode;
@ -397,7 +394,7 @@ _rawWriteExit:
#endif #endif
/* trim sent data from outBuf */ /* trim sent data from outBuf */
laikaM_rmvarray(sock->outBuf, sock->outCount, 0, sentBytes); laikaM_rmvVector(sock->outBuf, 0, sentBytes);
*processed = sentBytes; *processed = sentBytes;
return errCode; return errCode;

View File

@ -1,10 +1,11 @@
#ifndef SHELLCLIENT_H #ifndef SHELLCLIENT_H
#define SHELLCLIENT_H #define SHELLCLIENT_H
#include "hashmap.h" #include "core/hashmap.h"
#include "lpeer.h" #include "core/lmem.h"
#include "lsodium.h" #include "core/lsodium.h"
#include "ltask.h" #include "core/ltask.h"
#include "net/lpeer.h"
#include "speer.h" #include "speer.h"
typedef struct sShell_client typedef struct sShell_client
@ -15,9 +16,7 @@ typedef struct sShell_client
struct sLaika_peer *peer; struct sLaika_peer *peer;
tShell_peer *openShell; /* if not NULL, shell is open on peer */ tShell_peer *openShell; /* if not NULL, shell is open on peer */
struct hashmap *peers; struct hashmap *peers;
tShell_peer **peerTbl; laikaM_newVector(tShell_peer *, peerTbl);
int peerTblCount;
int peerTblCap;
} tShell_client; } tShell_client;
#define shellC_isShellOpen(x) (x->openShell != NULL) #define shellC_isShellOpen(x) (x->openShell != NULL)

View File

@ -1,8 +1,8 @@
#ifndef SHELLPEER_H #ifndef SHELLPEER_H
#define SHELLPEER_H #define SHELLPEER_H
#include "lpeer.h" #include "core/lsodium.h"
#include "lsodium.h" #include "net/lpeer.h"
typedef struct sShell_peer typedef struct sShell_peer
{ {

View File

@ -1,4 +1,4 @@
#include "ini.h" #include "core/ini.h"
#include "sclient.h" #include "sclient.h"
#include "sterm.h" #include "sterm.h"

View File

@ -1,9 +1,9 @@
#include "sclient.h" #include "sclient.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
#include "lpacket.h" #include "core/lsodium.h"
#include "lsodium.h" #include "net/lpacket.h"
#include "sterm.h" #include "sterm.h"
void shell_pingTask(struct sLaika_taskService *service, struct sLaika_task *task, clock_t currTick, void shell_pingTask(struct sLaika_taskService *service, struct sLaika_task *task, clock_t currTick,
@ -217,9 +217,7 @@ void shellC_init(tShell_client *client)
client->peers = hashmap_new(sizeof(tShell_hashMapElem), 8, 0, 0, shell_ElemHash, client->peers = hashmap_new(sizeof(tShell_hashMapElem), 8, 0, 0, shell_ElemHash,
shell_ElemCompare, NULL, NULL); shell_ElemCompare, NULL, NULL);
client->openShell = NULL; client->openShell = NULL;
client->peerTbl = NULL; laikaM_initVector(client->peerTbl, 4);
client->peerTblCap = 4;
client->peerTblCount = 0;
laikaT_initTaskService(&client->tService); laikaT_initTaskService(&client->tService);
laikaT_newTask(&client->tService, LAIKA_PING_INTERVAL, shell_pingTask, client); laikaT_newTask(&client->tService, LAIKA_PING_INTERVAL, shell_pingTask, client);
@ -254,7 +252,7 @@ void shellC_cleanup(tShell_client *client)
laikaT_cleanTaskService(&client->tService); laikaT_cleanTaskService(&client->tService);
/* free peers */ /* free peers */
for (i = 0; i < client->peerTblCount; i++) { for (i = 0; i < laikaM_countVector(client->peerTbl); i++) {
if (client->peerTbl[i]) if (client->peerTbl[i])
shellP_freePeer(client->peerTbl[i]); shellP_freePeer(client->peerTbl[i]);
} }
@ -346,16 +344,15 @@ int shellC_addPeer(tShell_client *client, tShell_peer *newPeer)
{ {
/* find empty ID */ /* find empty ID */
int id; int id;
for (id = 0; id < client->peerTblCount; id++) { for (id = 0; id < laikaM_countVector(client->peerTbl); id++) {
if (client->peerTbl[id] == NULL) /* it's empty! */ if (client->peerTbl[id] == NULL) /* it's empty! */
break; break;
} }
/* if we didn't find an empty id, grow the array */ /* if we didn't find an empty id, grow the array (ID is already set to the correct index) */
if (id == client->peerTblCount) { if (id == laikaM_countVector(client->peerTbl)) {
laikaM_growarray(tShell_peer *, client->peerTbl, 1, client->peerTblCount, laikaM_growVector(tShell_peer *, client->peerTbl, 1);
client->peerTblCap); laikaM_countVector(client->peerTbl)++;
client->peerTblCount++;
} }
/* add to peer lookup table */ /* add to peer lookup table */

View File

@ -1,7 +1,7 @@
#include "scmd.h" #include "scmd.h"
#include "lerror.h" #include "core/lerror.h"
#include "lmem.h" #include "core/lmem.h"
#include "sclient.h" #include "sclient.h"
#include "speer.h" #include "speer.h"
#include "sterm.h" #include "sterm.h"
@ -23,7 +23,7 @@ tShell_cmdDef *shellS_findCmd(char *cmd);
tShell_peer *shellS_getPeer(tShell_client *client, int id) tShell_peer *shellS_getPeer(tShell_client *client, int id)
{ {
if (id < 0 || id >= client->peerTblCount || client->peerTbl[id] == NULL) if (id < 0 || id >= laikaM_countVector(client->peerTbl) || client->peerTbl[id] == NULL)
CMD_ERROR("Not a valid peer ID! [%d]\n", id); CMD_ERROR("Not a valid peer ID! [%d]\n", id);
return client->peerTbl[id]; return client->peerTbl[id];
@ -48,7 +48,7 @@ void listPeersCMD(tShell_client *client, int argc, char *argv[])
{ {
int i; int i;
for (i = 0; i < client->peerTblCount; i++) { for (i = 0; i < laikaM_countVector(client->peerTbl); i++) {
if (client->peerTbl[i]) { if (client->peerTbl[i]) {
shellT_printf("%04d ", i); shellT_printf("%04d ", i);
shellP_printInfo(client->peerTbl[i]); shellP_printInfo(client->peerTbl[i]);
@ -172,11 +172,10 @@ void shellS_cleanupCmds(void)
char **shellS_splitCmd(char *cmd, int *argSize) char **shellS_splitCmd(char *cmd, int *argSize)
{ {
int argCount = 0;
int argCap = 4;
char *temp; char *temp;
char **args = NULL;
char *arg = cmd; char *arg = cmd;
laikaM_newVector(char *, args);
laikaM_initVector(args, 4);
do { do {
/* replace space with NULL terminator */ /* replace space with NULL terminator */
@ -192,12 +191,12 @@ char **shellS_splitCmd(char *cmd, int *argSize)
*arg++ = '\0'; *arg++ = '\0';
} }
/* insert into our 'args' array */ /* insert into our 'args' vector */
laikaM_growarray(char *, args, 1, argCount, argCap); laikaM_growVector(char *, args, 1);
args[argCount++] = arg; args[laikaM_countVector(args)++] = arg;
} while ((arg = strchr(arg, ' ')) != NULL); /* while we still have a delimiter */ } while ((arg = strchr(arg, ' ')) != NULL); /* while we still have a delimiter */
*argSize = argCount; *argSize = laikaM_countVector(args);
return args; return args;
} }

View File

@ -1,7 +1,7 @@
#include "speer.h" #include "speer.h"
#include "lmem.h" #include "core/lmem.h"
#include "lpacket.h" #include "net/lpacket.h"
#include "sterm.h" #include "sterm.h"
tShell_peer *shellP_newPeer(PEERTYPE type, OSTYPE osType, uint8_t *pubKey, char *hostname, tShell_peer *shellP_newPeer(PEERTYPE type, OSTYPE osType, uint8_t *pubKey, char *hostname,

View File

@ -1,6 +1,6 @@
#include "sterm.h" #include "sterm.h"
#include "lmem.h" #include "core/lmem.h"
#include "scmd.h" #include "scmd.h"
#define KEY_ESCAPE 0x001b #define KEY_ESCAPE 0x001b
@ -15,8 +15,42 @@
#define cursorBackward(x) printf("\033[%dD", (x)) #define cursorBackward(x) printf("\033[%dD", (x))
#define clearLine() printf("\033[2K") #define clearLine() printf("\033[2K")
/* =================================[[ DEPRECATED ARRAY API ]]================================== */
/*
this whole target needs to be rewritten, so these macros have been embedded here until a
rewrite can be done. this is the only section of the entire codebase that relies too heavily
on these to quickly exchange into the vector api equivalent. there's technically a memory leak
here since the array is never free'd, but since the array is expected to live the entire
lifetime of the program it's safe to leave as-is for now.
*/
#define laikaM_growarray(type, buf, needed, count, capacity) \
if (count + needed >= capacity || buf == NULL) { \
capacity = (capacity + needed) * GROW_FACTOR; \
buf = (type *)laikaM_realloc(buf, sizeof(type) * capacity); \
}
/* moves array elements above indx down by numElem, removing numElem elements at indx */
#define laikaM_rmvarray(buf, count, indx, numElem) \
do { \
int _i, _sz = ((count - indx) - numElem); \
for (_i = 0; _i < _sz; _i++) \
buf[indx + _i] = buf[indx + numElem + _i]; \
count -= numElem; \
} while (0);
/* moves array elements above indx up by numElem, inserting numElem elements at indx */
#define laikaM_insertarray(buf, count, indx, numElem) \
do { \
int _i; \
for (_i = count; _i > indx; _i--) \
buf[_i] = buf[_i - 1]; \
count += numElem; \
} while (0);
struct termios orig_termios; struct termios orig_termios;
char *cmd, *prompt = "$> "; char *cmd = NULL, *prompt = "$> ";
int cmdCount = 0, cmdCap = 4, cmdCursor = 0; int cmdCount = 0, cmdCap = 4, cmdCursor = 0;
void shellT_conioTerm(void) void shellT_conioTerm(void)

View File

@ -1,5 +1,5 @@
#include "lerror.h" #include "core/lerror.h"
#include "lsodium.h" #include "core/lsodium.h"
#include <stdio.h> #include <stdio.h>
#include <string.h> #include <string.h>

View File

@ -1,5 +1,5 @@
#include "lbox.h" #include "core/lbox.h"
#include "lvm.h" #include "core/lvm.h"
#include <stdio.h> #include <stdio.h>
#include <string.h> #include <string.h>