Dat:// and React Native

Recently I wrote about our struggles to use dat:// with react-native. As a follow-up, I would like to start this meta-thread for getting dat:// to run smoothly on/with React-native.

→ If you are interesting in supporting and/or collaborating this please comment and share your struggles or success stories! :hugs:

I believe that there are a few teams here that try to get dat:// to run on mobile devices and together we should be able to make this a lot easier. :rocket::first_quarter_moon_with_face:


↓ Geeky & detailed problem description follows below :nerd_face:


Thinking about what could be done, basically we have identified two different aims:

  1. Getting dat:// to run in a regular react-native environment
  2. Getting dat:// to run without native extensions (javascript-only) to be used within an expo workflow.

These are fundamentally different aims as expo does not support native extensions beyond those bundled with the expo test app. This means…

The impact of this is that for expo and the foreseeable future, we need to use WebSockets or maybe expo#webrtc (please vote for this) to connect to the dat:// network. Also, either all packages need to be adjusted to work within the limitations of the expo build system (e.g. use the buffer npm package or a working version of ReactNativify-setup needs to be prepared to run dat out of the box.

For both approaches we have identified these first general issues (please comment if you find others):

  • Writing to the filesystem on mobile devices can be a bottleneck. Aside from hardware limits, the expo file API is very rudimentary and getting a well-performance random-access-* system working might be a challenge.
  • If you have special network settings, those need to be set for every mobile app. It would be good to have standard settings that could be set-up once and work for every mobile app (the same way).
  • Displaying and controlling media assets in mobile phones is (much more than on desktop) working through threads. Using the react-native-nodejs message API, the image/movie assets need to be sent using strings which will be passed between two threads. String/Buffer transformations are heavy which will certainly make image display not quick. This is exasperated to the issue that react-native-fast-image is not added to expo (vote for it → expo#add-react-native-fast-image).
  • Dealing with the app running in background is very tricky. Should the connection be dropped? Should it be kept alive for a little and dropped after a while? It would be good to have best practices here.
  • React-Native does not support WASM/Webassembly (vote → react-native#support-wasmwebassembly), which means modules in the dat:// ecosystem would need to continue to work without using wasm. There are workarounds using Webassembly.js, but they bring problems: further reading: [1], [2]

4 Likes

Nice writeup, Martin. Out of curiosity, if there were a Rust or C++ impl of the stack, which of these limitations still apply?

If a Rust or C++ implementation was available, the question transforms to: Where to put the business logic of the application and what needs to be presented in the view layer?

The advantage of react-native is that the we can write the business logic in JavaScript. Given that premise and that the rust context runs in a separate thread it would mean that we need a RPC API for communicating with rust. I support having this sort of API but it is rather inflexible and transferring data like images or videos from the rust process to the visual components in the webview might simply be slow or tricky without fully integrated support.

More importantly to me and to people starting out with react native however is that a rust implementation has almost no chance to ever run in expo as I don’t see a reason for expo team to bundle the rust implementation in their client and they have been hesitant to bundle much more general and needed native implementations in the past.

If you change to premise to also write the business logic in rust then the question of “why react-native” comes to mind.

is react already fixed? because someone mentioned framework7 in the chat. does react offer a web-app variant in addition to ios and android?
at framework7 these three variants would be all in one and cordova works too. or is it just a mobile application?

1 Like

Hello Alex, thank you for chiming in! :hugs: In this thread I want to focus on how to get dat to work in react-native. That doesn’t mean that react-native is the best or only platform. To get dat to work with other platforms I think we should have separate threads. If you feel like it and have the time: I would very much appreciate a post on how to get dat:// to work smoothly with another platform (like framework7).

Hi Martin, thanks for great summary. At Cliqz we build browsers that aim to provide dat:// protocol support and due to work of @sammacbeth we can already do it on Desktop.

On mobile however, custom protocol integration is more complicated. Cliqz browser for Android is soon going to release new version based on GeckoView (Firefox), so it is likely we could take the same approach as Sam took on Desktop, on iOS unfortunately there are no WebExtensions.

Cliqz browser for iOS is built on top Firefox for iOS codebase extended with Cliqz shared code base running in React Native.

We would very much like to learn more about your experinece with running dat:// in React Native.

Even though we don’t have any POC on iOS we are somehow preparing for having one in the future.

As example of such preparations you can take a look on our implementation of window.crypto.suble APIs for React Native:


It was mentioned already that such APIs are suboptimal as require data serialization (thus additional allocaitons) over RN bridge.

Our hope is that upcomming React Native development called TurboModules will allow us to write synchornous native apis without redundand memory allocations. TurboModules however are still in active development and I could not find any documentation on how to use them.

Regarding libsodium synchronous API - IMAO the only resonable way forward is to assume that crypto APIs (and any other heavy operations) is always asynchronously.
Not sure how @sammacbeth overcame that problem on WebExtension port, but maybe same can be done for React Native.

If anybody in community is interested in helping Cliqz with dat:// support for iOS please feel free to contact us on Github. All help is very appriciated, the browser code is licensed under MPL-v2 (check the first link about crypto API above).

2 Likes

Hello @chrmod,

Thank you for following up after our last comm-comm conversation

preparing for having one in the future.

I saw https://github.com/cliqz/dat-react-native - great start for this!

Our hope is that upcomming React Native development called TurboModules will allow us to write synchornous native apis without redundand memory allocations.

Nice, I didn’t know about it. This seems to have been in the works for a while now. In react-anative/proposal#4 about fabric, there is the message:

start rolling it out to all of you in open source in the middle of 2020

Not sure how covid affected this timeline, but there is also this mention

we push for full rollout of TurboModules in H1 2020.

This would mean that we can have native code with shared memory, awesome! To me this means that trying to get sodium to run asynchronously is a temporary win that will not help us in the long run.