Commit Graph

333 Commits

Author SHA1 Message Date
Hamish Milne
e4f05884c3 Fixed serialization runtime exceptions 2020-02-13 17:42:08 +08:00
Hamish Milne
ca971ff31f Serialize file/directory services 2020-02-13 17:42:08 +08:00
Hamish Milne
9525d81344 More base-derived fixes 2020-02-13 17:42:07 +08:00
Hamish Milne
996aba39fe Correct exports; add some file serialization; fix service base object serialization 2020-02-13 17:42:07 +08:00
Hamish Milne
558e710e17 Finished archives; remove pod.h 2020-02-13 17:41:26 +08:00
Hamish Milne
2bf5b46460 Basic archive backend serialization 2020-02-13 17:41:26 +08:00
Hamish Milne
2d2c7218ef Serialize CECD, CFG services 2020-02-13 17:41:25 +08:00
zhupengfei
4273b967b5
core/file_sys: Do not apply the same mods to DLCs
Now you can apply separate mods to DLCs and mods for the original title won't be applied.
2020-02-11 14:03:07 +08:00
zhupengfei
d9ae4c332d
layered_fs: Do not open all replacement files on load
Instead open them when we want to read them. This is because the standard library has a limit on the number of opened files.
2020-02-09 21:48:42 +08:00
zhupengfei
13e2d534e9
core: Add dump RomFS support
This is added to LayeredFS, then the NCCH container and then the loader interface.
2020-02-09 20:59:31 +08:00
zhupengfei
db18f6c79a
Address review
simplify code
2020-02-07 23:53:00 +08:00
zhupengfei
6e0afbaa19
Fix build
Explicitly use `std::min<std::size_t>`

Added virtual destructor
2020-02-07 16:26:33 +08:00
zhupengfei
eed9de2336
core/file_sys: Allow exheader replacement to be read from mods path
The previous method (filename.exheader) can still be used.
2020-02-07 15:55:35 +08:00
zhupengfei
53d0c618a0
core/file_sys: Read mods for the original title for updates
Updates can override RomFS and ExeFS, therefore we should apply the mods to them as well.
2020-02-07 15:20:20 +08:00
zhupengfei
91e5a39a08
core/file_sys: Allow exefs mods to be read from mods path
The original path (file_name.exefsdir) is still supported, but alternatively users can choose to put exefs patches in the same place as LayeredFS files (`load/mods/<Title ID>/exefs`).
2020-02-07 15:20:19 +08:00
zhupengfei
8a570bf00c
core: Use LayeredFS while reading RomFS
Only enabled for NCCHs that do not have an override romfs.

LayeredFS files should be put in the `load` directory in User Directory. The directory structure is similar to yuzu's but currently does not allow named mods yet. Replacement files should be put in `load/mods/<Title ID>/romfs` while patches/stubs should be put in `load/mods/<Title ID>/romfs_ext`.
2020-02-07 15:20:18 +08:00
zhupengfei
890405bb7c
core/file_sys: LayeredFS implementation
This implementation is different from Luma3DS's which directly hooks the SDK functions. Instead, we read the RomFS's metadata and figure out the directory and file structure. Then, relocations (i.e. replacements/deletions/patches) are applied. Afterwards, we rebuild the metadata, and assign 'fake' data offsets to the files. When we want to read file data from this rebuilt RomFS, we use binary search to find the last data offset smaller or equal to the given offset and read from that file  (either from the original RomFS, or from replacement files, or from buffered data with patches applied) and any later files when length is not enough.

The code that rebuilds the metadata is pretty complex and uses quite a few variables to keep track of necessary information like metadata offsets. According to my tests, it is able to build RomFS-es identical to the original (but without trailing garbage data) when no relocations are applied.
2020-02-07 15:19:57 +08:00
zhupengfei
83e0cc45f4
core/file_sys: Make RomFSReader an abstract interface
The original RomFSReader is renamed to DirectRomFSReader that directly reads the RomFS.
2020-02-07 15:19:53 +08:00
Léo Lam
756d231ff9 file_sys: Add support for BPS patches
The BPS format allows distributing patches that are smaller and that do
not contain copyrighted content if data is relocated
(unlike non-trivial IPS patches).

This is essential for games such as MM3D that have three barely
different code revisions. Supporting all three versions would
demand an unreasonable amount of work; with BPS patches only one
version has to be supported.
2019-12-22 10:46:47 +01:00
Léo Lam
3140086c60 file_sys: Handle patch applying failures
This changes ApplyCodePatch to return a ResultStatus, which makes it
possible to determine whether patch applying has failed. Previously,
only a boolean was returned, and false was returned when no patch
was found OR when a patch was found but applying it failed.

This also changes AppLoader_NCCH to return an error if patching fails
because the executable is likely to be left in an inconsistent state
and we should not proceed booting in that case.
2019-12-22 10:46:47 +01:00
Léo Lam
1377be9902 file_sys: Move IPS patching code into separate source file
In anticipation of a new BPS patcher.
2019-12-22 10:46:47 +01:00
Léo Lam
7318913f5a ncch_container: Fix NCCH decryption heuristic when replacing exheader
Fixes a regression from #4862, which caused the NCCH title ID checking
heuristic to be skipped whenever the exheader is replaced.

I was thinking the heuristic wouldn't be needed in that case, but it
turns out that many users still have pathological NCCHs that indicate
they are encrypted but are actually decrypted...

Now the original exheader is always read and used for the heuristic
to determine whether the NCCH is actually encrypted; only then do we
load a replacement exheader (if it exists) to avoid affecting the
heuristic.
2019-08-22 17:52:30 +08:00
Léo Lam
a2cde6e075 ncch_container: Assume override exheaders are decrypted
Require and assume that override exheaders are decrypted for
consistency with Luma's loader behaviour and to ensure consistent
behaviour regardless of whether the NCCH is marked as encrypted or not.
Currently, exheader overriding with an encrypted NCCH would cause
the title ID checking heuristic to mistakenly disable encryption,
which would then make exefs loading fail.
2019-08-07 18:41:37 +08:00
James Rowe
aef06df388
Merge pull request #4817 from leoetlino/ips-patch-later
ncch_container: Apply IPS patch after allocating .bss
2019-07-23 20:58:35 -06:00
Weiyi Wang
909d04ddea Remove unused stuff
Some unused variables are still kept in services, as they are parameters passed from the command buffer and might be used in the future
2019-07-16 20:27:03 -04:00
Léo Lam
43d1ee6fb7 ncch_container: Apply IPS patch after allocating .bss
This allows an IPS patch to edit .bss. This is useful for game patches
that need to add code, as putting things in .bss allows adding new code
*without* editing .code and thus without having to relocate everything.
2019-07-10 12:57:30 +02:00
Léo Lam
bc30412484 ncch_container: Add support for exheader replacement
This adds support for custom exheaders, which brings feature parity
with Luma3DS's code patching functionality and allows for injecting
new game code.
2019-07-06 18:01:54 +02:00
Léo Lam
70fe0d22de ncch_container: Fix code.bin replacement
Currently, having a replacement code.bin causes Citra to crash as it is
trying to load icon data from code.bin because of a typo. This commit
fixes that typo.
2019-06-30 11:48:28 +02:00
B3n30
14ed702971 FileSys: Load the ticket when the CIAContainer is loaded 2019-04-28 21:21:28 +02:00
fearlessTobi
e9bd34f7da Address remaining review comments 2019-04-07 17:00:28 +02:00
fearlessTobi
781d4b787a Address first batch of review comments 2019-04-07 15:35:32 +02:00
Weiyi Wang
21bda754b3
Merge pull request #4396 from FearlessTobi/open-delays
fs_user: Add a delay for each file open
2019-03-23 12:21:12 -04:00
Weiyi Wang
36ae10f555 archive_ncch: add open source mii archive 2019-03-09 08:19:32 -05:00
fearlessTobi
2999c8325e archive_extsavedata: add correct FileOpen delays 2019-03-06 18:53:44 +01:00
fearlessTobi
5055a6c262 Address review comments 2019-03-06 18:53:34 +01:00
fearlessTobi
fc7e6c9cc9 fs_user: Add a delay for each file open 2019-03-06 16:09:24 +01:00
Weiyi Wang
9c57b74907
Merge pull request #4618 from wwylele/fs-clean
FS: pass down program ID for archive operation (cleanup System::GetInstance part 3)
2019-03-04 12:34:54 -05:00
tgsm
39e163b7ce core: use nested namespaces 2019-02-18 20:34:18 -05:00
Weiyi Wang
a6d9baa05a FS: pass down program ID for archive operation
To eliminate System::GetInstance usage. Archive type like SelfNCCH and SaveData changes the actual reference path for different client, so archive backend interface should accept client information from the service interface. Currently we only pass the program ID as the client information.
2019-02-02 20:44:13 -05:00
Weiyi Wang
104829ae58 remove all occurance of specifying endianness inside BitField
This commit it automatically generated by command in zsh:
sed -i -- 's/BitField<\(.*\)_le>/BitField<\1>/g' **/*(D.)

BitField is now aware to endianness and default to little endian. It expects a value representation type without storage specification for its template parameter.
2019-01-28 22:09:43 -05:00
Zak Kurka
1ded48f5a3 Merge branch 'master' of github.com:citra-emu/citra into ips-patches 2018-11-15 07:11:18 -06:00
Zak Kurka
896577044d Remove ApplyIPS from the class and header 2018-11-15 07:11:06 -06:00
Weiyi Wang
5179915fb4 filesys/ncch: prevent buffer overflow on calculating SHA256 2018-11-10 22:58:32 -05:00
Weiyi Wang
5b7d21c3cd FileSys/DelayGenerator: add missing #include and virtual dtor (#4363)
* FileSys/DelayGenerator: add missing #include and virtual dtor

Added the needed include so that it won't cause error if another file includes this without including the depended files

Deleting a virtual class via base type without virtual dtor is UB, which happens inFileBackend.

* FileSys/DelayGenerator: move function definition into cpp file/n/nTo avoid generating vtable in all units that includes the header file

* filesys/delay_generator: rearrange #include
2018-10-27 12:46:03 +08:00
Weiyi Wang
f3ee5feb02
Merge pull request #4358 from wwylele/kernel-global-2
kernel: Eliminate global state in process and handle_table
2018-10-26 15:51:36 -04:00
Weiyi Wang
fa0e82b812
SeedDB: replace seek(tell, set) with seek(cur) (#4344) 2018-10-25 21:39:26 -04:00
Weiyi Wang
8fb3d8ff38 kernel/process: move current process to kernel instance
Two functional change:
QueryProcessMemory uses the process passed from handle instead current_process
Thread::Stop() uses TLS from owner_process instead of current_process
2018-10-22 21:32:34 -04:00
Zak Kurka
92f8b71137 Make ApplyIPS static and private 2018-10-21 10:42:23 -05:00
Zak Kurka
05c1816504 Stay in bounds 2018-10-21 07:48:54 -05:00
Zak Kurka
1c62bc06cf Add rle record support 2018-10-21 07:48:29 -05:00
Zak Kurka
d8284dfacc Log invalid IPS attempt 2018-10-21 07:47:27 -05:00
Zak Kurka
d0b54e9939 Check for EOF 2018-10-20 23:25:16 -05:00
Zak Kurka
2778801dc5 Updated based on review 2018-10-20 19:45:16 -05:00
Zak Kurka
2a5b56bb1d Be more intentional with check 2018-10-20 18:17:54 -05:00
Zak Kurka
bf50bb06f1 Minor style changes and use memcpy 2018-10-20 18:10:32 -05:00
Zak Kurka
2496ecbaf6 Added IPS patching 2018-10-20 13:09:05 -05:00
Ben
b01b94d843 FileSys::Ticket::Load: Return error if signature type does not match (#4339)
* FileSys::Ticket::Load: Return error if signature type does not match

* fixup! FileSys::Ticket::Load: Return error if signature type does not match
2018-10-15 11:26:35 -04:00
Weiyi Wang
9adc407112
Merge pull request #4304 from B3n30/std_optional
Replace boost::optional with std::optional where possible
2018-10-11 12:40:00 -04:00
Weiyi Wang
756a3d404a
Merge pull request #4283 from valentinvanelslande/seed-db
Implement SeedDB & Seed Crypto
2018-10-08 14:18:42 -04:00
B3n30
d37a2270d6 Replace boost::optional with std::optional where possible 2018-10-05 13:51:09 +02:00
Pengfei Zhu
87e16c80ac
Merge pull request #4229 from zhaowenlan1779/open-folder
citra_qt, core: game list "Open XXX Location" improvements
2018-10-05 00:46:43 -05:00
Weiyi Wang
eb494c574d
Merge pull request #4244 from wwylele/swap-enum
common/swap: add swap template for enum
2018-10-04 14:51:49 -04:00
Valentin Vanelslande
b067b5aa5a Implement SeedDB & Seed Crypto 2018-10-02 10:12:40 -05:00
zhupengfei
8cf1132b2e
ncch_container: choose the first id as extdata id when extended save data access is used 2018-10-02 08:15:10 +08:00
Weiyi Wang
fceec34945
Merge pull request #4278 from wwylele/cfg-ptm-direct-fs
cfg, ptm: access FS via backend directly
2018-10-01 18:32:43 -04:00
Weiyi Wang
5fb3137bdc
Merge pull request #4181 from wwylele/cia-crypto
Add encrypted CIA support
2018-10-01 14:23:35 -04:00
zhupengfei
bbf391abb9
citra_qt, core: game list "Open XXX Location" improvements 2018-10-01 08:42:22 +08:00
Weiyi Wang
b03332d09b fs: remove ArchiveFactory_ExtSaveData::Initialize
This is unnecessary as any open archive would create the full path for themselves
2018-09-28 17:08:39 -04:00
Weiyi Wang
df77491938 FileSys/cia: add ticket parsing 2018-09-23 13:05:55 -04:00
Weiyi Wang
1123580738 archive_ncch: add open source country list archive 2018-09-22 16:11:57 -04:00
Weiyi Wang
39feb0610b filesys/archive_selfncch: specify endiannes for enum 2018-09-22 00:28:44 -04:00
Weiyi Wang
9564b4b292 filesys/archive_ncch: specify endianness for enum
NCCHArchivePath::media_type is unchanged because its underlying type is unclear
2018-09-22 00:28:44 -04:00
Weiyi Wang
7d8f115185 Prefix all size_t with std::
done automatically by executing regex replace `([^:0-9a-zA-Z_])size_t([^0-9a-zA-Z_])` -> `$1std::size_t$2`
2018-09-06 16:03:28 -04:00
fearlessTobi
cf8b1726df Address review comments 2018-09-06 00:00:21 +02:00
fearlessTobi
cb3ef488c7 Fixup! string_util: Remove StringFromFormat() and related functions 2018-09-06 00:00:21 +02:00
Lioncash
3284bef360 string_util: Remove StringFromFormat() and related functions
Given we utilize fmt, we don't need to provide our own functions for formatting anymore
2018-09-06 00:00:21 +02:00
B3n30
bfbb6f13b4 NCCH_COntainer: Add file_path log to Load() 2018-08-31 11:00:34 +02:00
Ben
5e658efdb8
Service::HTTP_C: Add decryption of the ClCertA (#4045)
* Service::HTTP_C: Add decryption of the ClCertA

* fixup! Service::HTTP_C: Add decryption of the ClCertA

* fixup! Service::HTTP_C: Add decryption of the ClCertA

* FileSys:: Add MakeNCCHArchivePath and MakeNCCHFilePath; Small fixes in HTTP_C::DecryptDefaultClientCert

* fixup! fixup! Service::HTTP_C: Add decryption of the ClCertA

* fixup! fixup! fixup! Service::HTTP_C: Add decryption of the ClCertA
2018-08-09 23:02:53 +02:00
wwylele
d4a808c885 ncch_container: support encrypted games 2018-08-07 18:53:06 +03:00
BreadFish64
74cd98ecad core: clean up warnings 2018-08-01 14:10:23 -05:00
B3n30
97fda30839 Use std::begin and std::end for system archives c array 2018-07-28 12:07:11 +02:00
B3n30
7605e26b94 Add virtual bad word list; Load if dump is missing 2018-07-22 22:55:44 +02:00
Ben
7f1303a834 Use open source Shared Font if no dumped file is available (#3881)
* Add virtual SharedFont; Load if dumped is missing

* Move open_source_archives to externals; add readme and git hash to open_source_archives

* Updated shared font to newest version: m fixed, symbols fixed
2018-07-18 10:20:31 -06:00
wwylele
0eab948728 reformat all files with clang-format 2018-06-29 16:56:12 +03:00
wwylele
7c5a76e58b log: replace all NGLOG with LOG 2018-06-29 14:18:07 +03:00
Daniel Lim Wee Soong
4d2cbf271c Fix wrongly replaced arguments 2018-06-09 16:35:37 +08:00
Daniel Lim Wee Soong
d81cacfb9e core/file_sys: Replace logging macros 2018-06-07 23:06:44 +08:00
Lioncash
0e7ca706dc disk_archive: Make DiskDirectory's constructor explicit
Prevents implicit conversions/construction.
2018-03-31 00:54:17 -04:00
Lioncash
07855c48d1 disk_archive: Remove unnecessary const in DiskDirectory's Read() declaration
This only applies in the definition of the function, not the
declaration.
2018-03-31 00:54:17 -04:00
Lioncash
ed64d15060 disk_archive: Remove unused total_entries_in_directory member from DiskDirectory
Also converts initializer list assignments to in-class initializers
where relevant.
2018-03-31 00:52:47 -04:00
Weiyi Wang
1c4b230346
Merge pull request #3604 from lioncash/ptr
archive_systemsavedata: Remove pointer cast from GetSystemSaveDataPath()
2018-03-29 11:52:36 +03:00
Lioncash
26b1aec3c1 archive_systemsavedata: Remove pointer cast from GetSystemSaveDataPath()
These kinds of casts invoke undefined behavior due to alignment rules.

Use memcpy instead which always does the right thing.
2018-03-28 22:17:26 -04:00
Lioncash
3af976e41e ncch_container: Get rid of pointer casts in LZSS_GetDecompressedSize() and LZSS_Decompress()
These kinds of casts invoke undefined behavior, due to both the aliasing
rule, and from an alignment point-of-view. Use the way that's guaranteed
to always work instead.
2018-03-28 19:55:49 -04:00
James Rowe
f61141e86a Update the entire application to use the new clang format style 2018-03-09 10:54:43 -07:00
James Rowe
f3234db5e2
Merge pull request #3440 from B3n30/file_read_delay
File_Sys: Add a size dependent delay for each file read
2018-03-04 21:32:50 -07:00
James Rowe
739f8e5367 Fix clang format 2018-03-04 20:59:42 -07:00
B3n30
d8ac3a3435 Fix include 2018-02-24 14:46:42 +01:00
B3n30
06a7676ed1 Add DelayGenerator for all file backends 2018-02-24 14:15:57 +01:00
B3n30
58b16c5459 File_Sys: Add a size dependend delay for each file read 2018-02-24 14:15:57 +01:00
wwylele
71fac7bd72 FS: convert to service framework 2018-02-15 23:02:09 +02:00