From bddeafc46cf3877f05ee943b3cbdc81ae08e35ae Mon Sep 17 00:00:00 2001 From: jubb Date: Fri, 9 Apr 2021 13:43:47 +1000 Subject: [PATCH 01/31] feat: update the repo documentation readme and building file --- BUILDING.md | 15 +++------------ README.md | 6 ++++-- 2 files changed, 7 insertions(+), 14 deletions(-) diff --git a/BUILDING.md b/BUILDING.md index 142069ff01..5fa550b3f9 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -21,15 +21,6 @@ Ensure that the following packages are installed from the Android SDK manager: In Android studio, this can be done from the Quickstart panel, choose "Configure" then "SDK Manager". In the SDK Tools tab of the SDK Manager, make sure that the "Android Support Repository" is installed, and that the latest "Android SDK build-tools" are installed. Click "OK" to return to the Quickstart panel. You may also need to install API version 28 in the SDK platforms tab. -You will then need to clone and run `./gradlew install` on each of the following repositories IN ORDER: - -* https://github.com/loki-project/loki-messenger-android-curve-25519 -* https://github.com/loki-project/loki-messenger-android-protocol -* https://github.com/loki-project/loki-messenger-android-meta -* https://github.com/loki-project/session-android-service - -This installs these dependencies into a local Maven repository which the main Session Android repository will then draw from. - Setting up a development environment and building from Android Studio ------------------------------------ @@ -37,7 +28,7 @@ Setting up a development environment and building from Android Studio 1. Open Android Studio. On a new installation, the Quickstart panel will appear. If you have open projects, close them using "File > Close Project" to see the Quickstart panel. 2. From the Quickstart panel, choose "Checkout from Version Control" then "git". -3. Paste the URL for the session-android project when prompted (https://github.com/loki-project/session-android.git). +3. Paste the URL for the session-android project when prompted (https://github.com/oxen-io/session-android.git). 4. Android Studio should detect the presence of a project file and ask you whether to open it. Click "yes". 5. Default config options should be good enough. 6. Project initialization and building should proceed. @@ -49,7 +40,7 @@ The following steps should help you (re)build Session from the command line once 1. Checkout the session-android project source with the command: - git clone https://github.com/loki-project/session-android.git + git clone https://github.com/oxen-io/session-android.git 2. Make sure you have the [Android SDK](https://developer.android.com/sdk/index.html) installed. 3. Create a local.properties file at the root of your source checkout and add an sdk.dir entry to it. For example: @@ -58,7 +49,7 @@ The following steps should help you (re)build Session from the command line once 4. Execute Gradle: - ./gradlew build + ./gradlew :app:build Contributing code ----------------- diff --git a/README.md b/README.md index e7a7279ea6..b4cd95a484 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,19 @@ [Download on the Google Play Store](https://getsession.org/android) +[Download via F-Droid](https://fdroid.getsession.org/fdroid/repo?fingerprint=DB0E5297EB65CC22D6BD93C869943BDCFCB6A07DC69A48A0DD8C7BA698EC04E6) + [Grab the APK here](https://github.com/loki-project/session-android/releases/latest) ## Summary -Session integrates directly with [Loki Service Nodes](https://lokidocs.com/ServiceNodes/SNOverview/), which are a set of distributed, decentralized and Sybil resistant nodes. Service Nodes act as servers which store messages offline, and a set of nodes which allow for onion routing functionality obfuscating users' IP addresses. For a full understanding of how Session works, read the [Session Whitepaper](https://getsession.org/whitepaper). +Session integrates directly with [Oxen Service Nodes](https://docs.oxen.io/about-the-oxen-blockchain/oxen-service-nodes), which are a set of distributed, decentralized and Sybil resistant nodes. Service Nodes act as servers which store messages offline, and a set of nodes which allow for onion routing functionality obfuscating users' IP addresses. For a full understanding of how Session works, read the [Session Whitepaper](https://getsession.org/whitepaper). ![AndroidSession](https://i.imgur.com/0YC9TyI.png) ## Want to contribute? Found a bug or have a feature request? -Please search for any [existing issues](https://github.com/loki-project/session-android/issues) that describe your bugs in order to avoid duplicate submissions. Submissions can be made by making a pull request to our development branch. If you don't know where to start contributing, try reading the Github issues page for ideas. +Please search for any [existing issues](https://github.com/oxen-io/session-android/issues) that describe your bugs in order to avoid duplicate submissions. Submissions can be made by making a pull request to our `dev` branch. If you don't know where to start contributing, try reading the Github issues page for ideas. ## Build instructions From 57534d31e76a185b49621e66fbe76b24a9e3fbd2 Mon Sep 17 00:00:00 2001 From: jubb Date: Fri, 9 Apr 2021 15:44:39 +1000 Subject: [PATCH 02/31] refactor: replace fdroid deep-link with the fdroid session landing page --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b4cd95a484..5e41159c45 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ [Download on the Google Play Store](https://getsession.org/android) -[Download via F-Droid](https://fdroid.getsession.org/fdroid/repo?fingerprint=DB0E5297EB65CC22D6BD93C869943BDCFCB6A07DC69A48A0DD8C7BA698EC04E6) +Add the [F-Droid repo](https://fdroid.getsession.org/) [Grab the APK here](https://github.com/loki-project/session-android/releases/latest) From 3c3f6c25f1761c3b7b284cc0b757798b13b21a84 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 6 May 2021 15:46:22 +1000 Subject: [PATCH 03/31] Fix various bugs --- app/build.gradle | 2 +- .../database/helpers/SQLCipherOpenHelper.java | 10 ++++++++++ .../securesms/loki/database/LokiAPIDatabase.kt | 4 ++-- .../securesms/loki/protocol/MultiDeviceProtocol.kt | 2 +- .../main/java/org/session/libsession/snode/SnodeAPI.kt | 3 ++- 5 files changed, 16 insertions(+), 5 deletions(-) diff --git a/app/build.gradle b/app/build.gradle index f7e122af7a..c4c06a84c4 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -158,7 +158,7 @@ dependencies { testImplementation 'org.robolectric:shadows-multidex:4.2' } -def canonicalVersionCode = 154 +def canonicalVersionCode = 156 def canonicalVersionName = "1.10.0" def postFixSize = 10 diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java b/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java index dc7a0f7ee4..ae8a72f174 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java +++ b/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java @@ -55,6 +55,7 @@ public class SQLCipherOpenHelper extends SQLiteOpenHelper { private static final int lokiV21 = 42; private static final int lokiV22 = 43; private static final int lokiV23 = 44; + private static final int lokiV24 = 45; // Loki - onUpgrade(...) must be updated to use Loki version numbers if Signal makes any database changes private static final int DATABASE_VERSION = lokiV23; @@ -281,6 +282,15 @@ public class SQLCipherOpenHelper extends SQLiteOpenHelper { db.execSQL(LokiMessageDatabase.getUpdateMessageMappingTable()); } + if (oldVersion < lokiV24) { + String swarmTable = LokiAPIDatabase.Companion.getSwarmTable(); + String snodePoolTable = LokiAPIDatabase.Companion.getSnodePoolTable(); + db.execSQL("DROP TABLE " + swarmTable); + db.execSQL("DROP TABLE " + snodePoolTable); + db.execSQL(LokiAPIDatabase.getCreateSnodePoolTableCommand()); + db.execSQL(LokiAPIDatabase.getCreateSwarmTableCommand()); + } + db.setTransactionSuccessful(); } finally { db.endTransaction(); diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiAPIDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiAPIDatabase.kt index 53075bdae7..bc53e1534d 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiAPIDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiAPIDatabase.kt @@ -27,7 +27,7 @@ class LokiAPIDatabase(context: Context, helper: SQLCipherOpenHelper) : Database( private val timestamp = "timestamp" private val snode = "snode" // Snode pool - private val snodePoolTable = "loki_snode_pool_cache" + public val snodePoolTable = "loki_snode_pool_cache" private val dummyKey = "dummy_key" private val snodePool = "snode_pool_key" @JvmStatic val createSnodePoolTableCommand = "CREATE TABLE $snodePoolTable ($dummyKey TEXT PRIMARY KEY, $snodePool TEXT);" @@ -36,7 +36,7 @@ class LokiAPIDatabase(context: Context, helper: SQLCipherOpenHelper) : Database( private val indexPath = "index_path" @JvmStatic val createOnionRequestPathTableCommand = "CREATE TABLE $onionRequestPathTable ($indexPath TEXT PRIMARY KEY, $snode TEXT);" // Swarms - private val swarmTable = "loki_api_swarm_cache" + public val swarmTable = "loki_api_swarm_cache" private val swarmPublicKey = "hex_encoded_public_key" private val swarm = "swarm" @JvmStatic val createSwarmTableCommand = "CREATE TABLE $swarmTable ($swarmPublicKey TEXT PRIMARY KEY, $swarm TEXT);" diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/protocol/MultiDeviceProtocol.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/protocol/MultiDeviceProtocol.kt index f25833902e..c7ba42ecf5 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/protocol/MultiDeviceProtocol.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/protocol/MultiDeviceProtocol.kt @@ -17,7 +17,7 @@ object MultiDeviceProtocol { val userPublicKey = TextSecurePreferences.getLocalNumber(context) ?: return val lastSyncTime = TextSecurePreferences.getLastConfigurationSyncTime(context) val now = System.currentTimeMillis() - if (now - lastSyncTime < 2 * 24 * 60 * 60 * 1000) return + if (now - lastSyncTime < 7 * 24 * 60 * 60 * 1000) return val contacts = ContactUtilities.getAllContacts(context).filter { recipient -> !recipient.isBlocked && !recipient.name.isNullOrEmpty() && !recipient.isLocalNumber && recipient.address.serialize().isNotEmpty() }.map { recipient -> diff --git a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt index e7bd5ce9e9..a5094a63ee 100644 --- a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt +++ b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt @@ -33,7 +33,7 @@ object SnodeAPI { // Settings private val maxRetryCount = 6 - private val minimumSnodePoolCount = 24 + private val minimumSnodePoolCount = 12 private val minimumSwarmSnodeCount = 2 // Use port 4433 if the API level can handle the network security configuration and enforce pinned certificates private val seedNodePort = if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) 443 else 4433 @@ -92,6 +92,7 @@ object SnodeAPI { "method" to "get_n_service_nodes", "params" to mapOf( "active_only" to true, + "limit" to 256, "fields" to mapOf( "public_ip" to true, "storage_port" to true, "pubkey_x25519" to true, "pubkey_ed25519" to true ) ) ) From cc7e4701a3f226c2300b4dbe057cf5903731ade8 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 6 May 2021 16:03:45 +1000 Subject: [PATCH 04/31] Increment database version --- app/build.gradle | 2 +- .../securesms/database/helpers/SQLCipherOpenHelper.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/app/build.gradle b/app/build.gradle index c4c06a84c4..0b6754d2f9 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -158,7 +158,7 @@ dependencies { testImplementation 'org.robolectric:shadows-multidex:4.2' } -def canonicalVersionCode = 156 +def canonicalVersionCode = 157 def canonicalVersionName = "1.10.0" def postFixSize = 10 diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java b/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java index ae8a72f174..ef04bedbb7 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java +++ b/app/src/main/java/org/thoughtcrime/securesms/database/helpers/SQLCipherOpenHelper.java @@ -58,7 +58,7 @@ public class SQLCipherOpenHelper extends SQLiteOpenHelper { private static final int lokiV24 = 45; // Loki - onUpgrade(...) must be updated to use Loki version numbers if Signal makes any database changes - private static final int DATABASE_VERSION = lokiV23; + private static final int DATABASE_VERSION = lokiV24; private static final String DATABASE_NAME = "signal.db"; private final Context context; From 4fff5ac2dc2b03f4e363335c09a4a63a25195028 Mon Sep 17 00:00:00 2001 From: jubb Date: Fri, 7 May 2021 11:48:03 +1000 Subject: [PATCH 05/31] refactor: make storage reference jobId by string in deletion, don't persist jobs we are about to delete, delete jobs that fail to serialize from storage (probably from corrupt or moved data classes) in temporary message send jobs --- .../securesms/ApplicationContext.java | 1 - .../securesms/database/Storage.kt | 10 +++--- .../loki/database/SessionJobDatabase.kt | 21 ++++++++---- .../libsession/messaging/StorageProtocol.kt | 6 ++-- .../messaging/jobs/AttachmentUploadJob.kt | 2 +- .../libsession/messaging/jobs/JobQueue.kt | 34 ++++++++++++++----- 6 files changed, 49 insertions(+), 25 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java b/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java index 33e55e18d3..cc61634f9a 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java +++ b/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java @@ -103,7 +103,6 @@ import dagger.ObjectGraph; import kotlin.Unit; import kotlinx.coroutines.Job; import network.loki.messenger.BuildConfig; -import nl.komponents.kovenant.Kovenant; import static nl.komponents.kovenant.android.KovenantAndroid.startKovenant; import static nl.komponents.kovenant.android.KovenantAndroid.stopKovenant; diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt index 0f449c9f3a..0ab4824343 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt @@ -185,15 +185,15 @@ class Storage(context: Context, helper: SQLCipherOpenHelper) : Database(context, DatabaseFactory.getSessionJobDatabase(context).persistJob(job) } - override fun markJobAsSucceeded(job: Job) { - DatabaseFactory.getSessionJobDatabase(context).markJobAsSucceeded(job) + override fun markJobAsSucceeded(jobId: String) { + DatabaseFactory.getSessionJobDatabase(context).markJobAsSucceeded(jobId) } - override fun markJobAsFailed(job: Job) { - DatabaseFactory.getSessionJobDatabase(context).markJobAsFailed(job) + override fun markJobAsFailed(jobId: String) { + DatabaseFactory.getSessionJobDatabase(context).markJobAsFailed(jobId) } - override fun getAllPendingJobs(type: String): List { + override fun getAllPendingJobs(type: String): Map { return DatabaseFactory.getSessionJobDatabase(context).getAllPendingJobs(type) } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt index dda3b7d7eb..04edd3715e 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt @@ -4,6 +4,7 @@ import android.content.ContentValues import android.content.Context import net.sqlcipher.Cursor import org.session.libsession.messaging.jobs.* +import org.session.libsignal.utilities.logging.Log import org.thoughtcrime.securesms.database.Database import org.thoughtcrime.securesms.database.helpers.SQLCipherOpenHelper import org.thoughtcrime.securesms.jobmanager.impl.JsonDataSerializer @@ -30,19 +31,25 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa database.insertOrUpdate(sessionJobTable, contentValues, "$jobID = ?", arrayOf(jobID)) } - fun markJobAsSucceeded(job: Job) { - databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf(job.id)) + fun markJobAsSucceeded(jobId: String) { + databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf(jobId)) } - fun markJobAsFailed(job: Job) { - databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf(job.id)) + fun markJobAsFailed(jobId: String) { + databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf(jobId)) } - fun getAllPendingJobs(type: String): List { + fun getAllPendingJobs(type: String): Map { val database = databaseHelper.readableDatabase return database.getAll(sessionJobTable, "$jobType = ?", arrayOf(type)) { cursor -> - jobFromCursor(cursor) - } + val jobId = cursor.getString(jobID) + try { + jobId to jobFromCursor(cursor) + } catch (e: Exception) { + Log.e("Loki", "Error serializing Job of type $type",e) + jobId to null + } + }.toMap() } fun getAttachmentUploadJob(attachmentID: Long): AttachmentUploadJob? { diff --git a/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt b/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt index f64aa1a032..da604264d1 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt @@ -44,9 +44,9 @@ interface StorageProtocol { // Jobs fun persistJob(job: Job) - fun markJobAsSucceeded(job: Job) - fun markJobAsFailed(job: Job) - fun getAllPendingJobs(type: String): List + fun markJobAsSucceeded(jobId: String) + fun markJobAsFailed(jobId: String) + fun getAllPendingJobs(type: String): Map fun getAttachmentUploadJob(attachmentID: Long): AttachmentUploadJob? fun getMessageSendJob(messageSendJobID: String): MessageSendJob? fun resumeMessageSendJobIfNeeded(messageSendJobID: String) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt index f135178618..1d8b1a7170 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt @@ -108,7 +108,7 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess val messageSendJob = storage.getMessageSendJob(messageSendJobID) MessageSender.handleFailedMessageSend(this.message, e) if (messageSendJob != null) { - storage.markJobAsFailed(messageSendJob) + storage.markJobAsFailed(messageSendJobID) } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt index 9b1f65f4a7..85a1f33289 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt @@ -78,10 +78,24 @@ class JobQueue : JobDelegate { return } hasResumedPendingJobs = true - val allJobTypes = listOf(AttachmentDownloadJob.KEY, AttachmentDownloadJob.KEY, MessageReceiveJob.KEY, MessageSendJob.KEY, NotifyPNServerJob.KEY) + val allJobTypes = listOf(AttachmentDownloadJob.KEY, + AttachmentDownloadJob.KEY, + MessageReceiveJob.KEY, + MessageSendJob.KEY, + NotifyPNServerJob.KEY + ) allJobTypes.forEach { type -> val allPendingJobs = MessagingModuleConfiguration.shared.storage.getAllPendingJobs(type) - allPendingJobs.sortedBy { it.id }.forEach { job -> + val pendingJobs = mutableListOf() + for ((id, job) in allPendingJobs) { + if (job == null) { + // job failed to serialize, remove it from the DB + handleJobFailedPermanently(id) + } else { + pendingJobs.add(job) + } + } + pendingJobs.sortedBy { it.id }.forEach { job -> Log.i("Jobs", "Resuming pending job of type: ${job::class.simpleName}.") queue.offer(job) // Offer always called on unlimited capacity } @@ -89,17 +103,18 @@ class JobQueue : JobDelegate { } override fun handleJobSucceeded(job: Job) { - MessagingModuleConfiguration.shared.storage.markJobAsSucceeded(job) + val jobId = job.id ?: return + MessagingModuleConfiguration.shared.storage.markJobAsSucceeded(jobId) } override fun handleJobFailed(job: Job, error: Exception) { job.failureCount += 1 val storage = MessagingModuleConfiguration.shared.storage if (storage.isJobCanceled(job)) { return Log.i("Jobs", "${job::class.simpleName} canceled.")} - storage.persistJob(job) if (job.failureCount == job.maxFailureCount) { - storage.markJobAsFailed(job) + handleJobFailedPermanently(job, error) } else { + storage.persistJob(job) val retryInterval = getRetryInterval(job) Log.i("Jobs", "${job::class.simpleName} failed; scheduling retry (failure count is ${job.failureCount}).") timer.schedule(delay = retryInterval) { @@ -110,10 +125,13 @@ class JobQueue : JobDelegate { } override fun handleJobFailedPermanently(job: Job, error: Exception) { - job.failureCount += 1 + val jobId = job.id ?: return + handleJobFailedPermanently(jobId) + } + + private fun handleJobFailedPermanently(jobId: String) { val storage = MessagingModuleConfiguration.shared.storage - storage.persistJob(job) - storage.markJobAsFailed(job) + storage.markJobAsFailed(jobId) } private fun getRetryInterval(job: Job): Long { From ccd9493f628407ad08e3cfd5aa427cb721b85573 Mon Sep 17 00:00:00 2001 From: jubb Date: Fri, 7 May 2021 12:02:12 +1000 Subject: [PATCH 06/31] refactor: remove unlimited by array size and cap at 10_000_000 --- .../org/session/libsession/messaging/jobs/MessageSendJob.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index c695df8311..de25a6fbb8 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -79,7 +79,7 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { override fun serialize(): Data { val kryo = Kryo() kryo.isRegistrationRequired = false - val output = Output(ByteArray(4096), -1) // maxBufferSize '-1' will dynamically grow internally if we run out of room serializing the message + val output = Output(ByteArray(4096), 10_000_000) // maxBufferSize '-1' will dynamically grow internally if we run out of room serializing the message kryo.writeClassAndObject(output, message) output.close() val serializedMessage = output.toBytes() From d707433f28a1d20e2c88dd19ed8dae823fc75f1c Mon Sep 17 00:00:00 2001 From: jubb Date: Fri, 7 May 2021 12:03:04 +1000 Subject: [PATCH 07/31] docs: remove no long applicable docs --- .../org/session/libsession/messaging/jobs/MessageSendJob.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index de25a6fbb8..83822c4fc7 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -79,7 +79,7 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { override fun serialize(): Data { val kryo = Kryo() kryo.isRegistrationRequired = false - val output = Output(ByteArray(4096), 10_000_000) // maxBufferSize '-1' will dynamically grow internally if we run out of room serializing the message + val output = Output(ByteArray(4096), 10_000_000) kryo.writeClassAndObject(output, message) output.close() val serializedMessage = output.toBytes() From e7377d640f91f3fb75b9b726587ec1f718356ba5 Mon Sep 17 00:00:00 2001 From: jubb Date: Fri, 7 May 2021 13:36:35 +1000 Subject: [PATCH 08/31] fix: use AttachmentUploadJob instead of two download job keys in all types --- .../main/java/org/session/libsession/messaging/jobs/JobQueue.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt index 85a1f33289..cffb2db7d6 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt @@ -78,7 +78,7 @@ class JobQueue : JobDelegate { return } hasResumedPendingJobs = true - val allJobTypes = listOf(AttachmentDownloadJob.KEY, + val allJobTypes = listOf(AttachmentUploadJob.KEY, AttachmentDownloadJob.KEY, MessageReceiveJob.KEY, MessageSendJob.KEY, From 11a89c0a76a6d4d3a666c78090d307330ac89ada Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Fri, 7 May 2021 16:30:52 +1000 Subject: [PATCH 09/31] Update version number --- app/build.gradle | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/build.gradle b/app/build.gradle index 0b6754d2f9..2888a2ba07 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -158,8 +158,8 @@ dependencies { testImplementation 'org.robolectric:shadows-multidex:4.2' } -def canonicalVersionCode = 157 -def canonicalVersionName = "1.10.0" +def canonicalVersionCode = 158 +def canonicalVersionName = "1.10.1" def postFixSize = 10 def abiPostFix = ['armeabi-v7a' : 1, From 8439d5711523305f378ba41c0abb75fbe0cd9301 Mon Sep 17 00:00:00 2001 From: jubb Date: Mon, 10 May 2021 17:07:10 +1000 Subject: [PATCH 10/31] refactor: let the periodic work run more frequently and never fail from excessive retries preventing from re-running. remove resume pending jobs from ApplicationContext onCreate and handle in home activity's onCreate instead. prevent some illegal argument exceptions from Random.kt by returning null if empty --- .../securesms/ApplicationContext.java | 2 -- .../loki/api/BackgroundPollWorker.kt | 29 ++++--------------- .../securesms/loki/api/PublicChatManager.kt | 5 +++- .../libsession/snode/utilities/Random.kt | 1 + .../session/libsignal/service/loki/Random.kt | 1 + 5 files changed, 11 insertions(+), 27 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java b/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java index cc61634f9a..64a9929630 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java +++ b/app/src/main/java/org/thoughtcrime/securesms/ApplicationContext.java @@ -327,7 +327,6 @@ public class ApplicationContext extends MultiDexApplication implements Dependenc .setJobStorage(new FastJobStorage(DatabaseFactory.getJobDatabase(this))) .setDependencyInjector(this) .build()); - JobQueue.getShared().resumePendingJobs(); } private void initializeDependencyInjection() { @@ -455,7 +454,6 @@ public class ApplicationContext extends MultiDexApplication implements Dependenc poller.setUserPublicKey(userPublicKey); return; } - LokiAPIDatabase apiDB = DatabaseFactory.getLokiAPIDatabase(this); poller = new Poller(); closedGroupPoller = new ClosedGroupPoller(); } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt index 7b4f2c2aa6..c020b5333d 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt @@ -17,7 +17,6 @@ import org.session.libsession.snode.SnodeAPI import org.session.libsession.utilities.TextSecurePreferences import org.session.libsignal.utilities.logging.Log import org.thoughtcrime.securesms.database.DatabaseFactory -import java.io.IOException import java.util.concurrent.TimeUnit class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Worker(context, params) { @@ -25,26 +24,10 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor companion object { const val TAG = "BackgroundPollWorker" - private const val RETRY_ATTEMPTS = 3 - - @JvmStatic - fun scheduleInstant(context: Context) { - val workRequest = OneTimeWorkRequestBuilder() - .setConstraints(Constraints.Builder() - .setRequiredNetworkType(NetworkType.CONNECTED) - .build() - ) - .build() - - WorkManager - .getInstance(context) - .enqueue(workRequest) - } - @JvmStatic fun schedulePeriodic(context: Context) { Log.v(TAG, "Scheduling periodic work.") - val workRequest = PeriodicWorkRequestBuilder(15, TimeUnit.MINUTES) + val workRequest = PeriodicWorkRequestBuilder(5, TimeUnit.MINUTES) .setConstraints(Constraints.Builder() .setRequiredNetworkType(NetworkType.CONNECTED) .build() @@ -55,7 +38,7 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor .getInstance(context) .enqueueUniquePeriodicWork( TAG, - ExistingPeriodicWorkPolicy.KEEP, + ExistingPeriodicWorkPolicy.REPLACE, workRequest ) } @@ -105,9 +88,8 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor return Result.success() } catch (exception: Exception) { - Log.v(TAG, "Background poll failed due to error: ${exception.message}.", exception) - - return if (runAttemptCount < RETRY_ATTEMPTS) Result.retry() else Result.failure() + Log.e(TAG, "Background poll failed due to error: ${exception.message}.", exception) + return Result.retry() } } @@ -116,8 +98,7 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor override fun onReceive(context: Context, intent: Intent) { if (intent.action == Intent.ACTION_BOOT_COMPLETED) { Log.v(TAG, "Boot broadcast caught.") - BackgroundPollWorker.scheduleInstant(context) - BackgroundPollWorker.schedulePeriodic(context) + schedulePeriodic(context) } } } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/api/PublicChatManager.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/api/PublicChatManager.kt index 04ed0e4c95..22878d46fd 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/api/PublicChatManager.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/api/PublicChatManager.kt @@ -31,7 +31,7 @@ class PublicChatManager(private val context: Context) { refreshChatsAndPollers() for ((threadID, _) in chats) { val poller = pollers[threadID] - areAllCaughtUp = if (poller != null) areAllCaughtUp && poller.isCaughtUp else true + areAllCaughtUp = if (poller != null) areAllCaughtUp && poller.isCaughtUp else areAllCaughtUp } return areAllCaughtUp } @@ -42,6 +42,9 @@ class PublicChatManager(private val context: Context) { val poller = pollers[threadID] ?: OpenGroupPoller(chat, executorService) poller.isCaughtUp = false } + for ((_,poller) in v2Pollers) { + poller.isCaughtUp = false + } } public fun startPollersIfNeeded() { diff --git a/libsession/src/main/java/org/session/libsession/snode/utilities/Random.kt b/libsession/src/main/java/org/session/libsession/snode/utilities/Random.kt index 2ec42cdf5b..72ceee9f3b 100644 --- a/libsession/src/main/java/org/session/libsession/snode/utilities/Random.kt +++ b/libsession/src/main/java/org/session/libsession/snode/utilities/Random.kt @@ -6,6 +6,7 @@ import java.security.SecureRandom * Uses `SecureRandom` to pick an element from this collection. */ fun Collection.getRandomElementOrNull(): T? { + if (isEmpty()) return null val index = SecureRandom().nextInt(size) // SecureRandom() should be cryptographically secure return elementAtOrNull(index) } diff --git a/libsignal/src/main/java/org/session/libsignal/service/loki/Random.kt b/libsignal/src/main/java/org/session/libsignal/service/loki/Random.kt index 68bc4380c5..b1c1cd2af7 100644 --- a/libsignal/src/main/java/org/session/libsignal/service/loki/Random.kt +++ b/libsignal/src/main/java/org/session/libsignal/service/loki/Random.kt @@ -6,6 +6,7 @@ import java.security.SecureRandom * Uses `SecureRandom` to pick an element from this collection. */ fun Collection.getRandomElementOrNull(): T? { + if (isEmpty()) return null val index = SecureRandom().nextInt(size) // SecureRandom() should be cryptographically secure return elementAtOrNull(index) } From 9f099771605cc528a333b88d72e778ca3e3c3589 Mon Sep 17 00:00:00 2001 From: jubb Date: Wed, 12 May 2021 10:43:17 +1000 Subject: [PATCH 11/31] refactor: remove registration required for job serialization and test logs, don't try to read class object if the message send class is not of expected type --- .../securesms/loki/api/SessionProtocolImpl.kt | 1 - .../libsession/messaging/jobs/AttachmentUploadJob.kt | 1 + .../java/org/session/libsession/messaging/jobs/Job.kt | 1 + .../org/session/libsession/messaging/jobs/JobQueue.kt | 1 + .../session/libsession/messaging/jobs/MessageSendJob.kt | 9 ++++++++- .../libsession/messaging/jobs/NotifyPNServerJob.kt | 1 + .../sending_receiving/ReceivedMessageHandler.kt | 1 - .../main/java/org/session/libsession/snode/SnodeAPI.kt | 2 +- 8 files changed, 13 insertions(+), 4 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/api/SessionProtocolImpl.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/api/SessionProtocolImpl.kt index 9be7b3e461..4916ef483d 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/api/SessionProtocolImpl.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/api/SessionProtocolImpl.kt @@ -23,7 +23,6 @@ class SessionProtocolImpl(private val context: Context) : SessionProtocol { override fun decrypt(ciphertext: ByteArray, x25519KeyPair: ECKeyPair): Pair { val recipientX25519PrivateKey = x25519KeyPair.privateKey.serialize() val recipientX25519PublicKey = Hex.fromStringCondensed(x25519KeyPair.hexEncodedPublicKey.removing05PrefixIfNeeded()) - Log.d("Test", "recipientX25519PublicKey: $recipientX25519PublicKey") val signatureSize = Sign.BYTES val ed25519PublicKeySize = Sign.PUBLICKEYBYTES diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt index 1d8b1a7170..c6e73e12e1 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt @@ -135,6 +135,7 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess override fun create(data: Data): AttachmentUploadJob { val serializedMessage = data.getByteArray(KEY_MESSAGE) val kryo = Kryo() + kryo.isRegistrationRequired = false val input = Input(serializedMessage) val message: Message = kryo.readObject(input, Message::class.java) input.close() diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt index 4693fddf4a..aefe7cc907 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt @@ -11,6 +11,7 @@ interface Job { // Keys used for database storage private val KEY_ID = "id" private val KEY_FAILURE_COUNT = "failure_count" + internal const val MAX_BUFFER_SIZE = 1_000_000 // bytes } fun execute() diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt index cffb2db7d6..ba21280700 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt @@ -50,6 +50,7 @@ class JobQueue : JobDelegate { private fun Job.canExecuteParallel(): Boolean { return this.javaClass in arrayOf( + MessageSendJob::class.java, AttachmentUploadJob::class.java, AttachmentDownloadJob::class.java ) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index 83822c4fc7..7b64f6bb77 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -4,6 +4,7 @@ import com.esotericsoftware.kryo.Kryo import com.esotericsoftware.kryo.io.Input import com.esotericsoftware.kryo.io.Output import org.session.libsession.messaging.MessagingModuleConfiguration +import org.session.libsession.messaging.jobs.Job.Companion.MAX_BUFFER_SIZE import org.session.libsession.messaging.messages.Destination import org.session.libsession.messaging.messages.Message import org.session.libsession.messaging.messages.visible.VisibleMessage @@ -79,7 +80,7 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { override fun serialize(): Data { val kryo = Kryo() kryo.isRegistrationRequired = false - val output = Output(ByteArray(4096), 10_000_000) + val output = Output(ByteArray(4096), MAX_BUFFER_SIZE) kryo.writeClassAndObject(output, message) output.close() val serializedMessage = output.toBytes() @@ -102,7 +103,13 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { val serializedMessage = data.getByteArray(KEY_MESSAGE) val serializedDestination = data.getByteArray(KEY_DESTINATION) val kryo = Kryo() + kryo.isRegistrationRequired = false var input = Input(serializedMessage) + val messageClass = kryo.readClass(input) + if (messageClass == null || !Message::class.java.isAssignableFrom(messageClass.type)) { + // if the message class doesn't exist or it doesn't implement `Message` parent class + throw Exception("deserialized messageClass was ${messageClass.type}") + } val message = kryo.readClassAndObject(input) as Message input.close() input = Input(serializedDestination) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt index fb99f54f56..720dd091ac 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt @@ -80,6 +80,7 @@ class NotifyPNServerJob(val message: SnodeMessage) : Job { override fun create(data: Data): NotifyPNServerJob { val serializedMessage = data.getByteArray(KEY_MESSAGE) val kryo = Kryo() + kryo.isRegistrationRequired = false val input = Input(serializedMessage) val message: SnodeMessage = kryo.readObject(input, SnodeMessage::class.java) input.close() diff --git a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt index 2a0b13ae3e..c63e86e56e 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt @@ -1,7 +1,6 @@ package org.session.libsession.messaging.sending_receiving import android.text.TextUtils -import okhttp3.HttpUrl import org.session.libsession.messaging.MessagingModuleConfiguration import org.session.libsession.messaging.jobs.AttachmentDownloadJob import org.session.libsession.messaging.jobs.JobQueue diff --git a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt index a5094a63ee..cb0d5ee837 100644 --- a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt +++ b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt @@ -92,7 +92,7 @@ object SnodeAPI { "method" to "get_n_service_nodes", "params" to mapOf( "active_only" to true, - "limit" to 256, +// "limit" to 256, "fields" to mapOf( "public_ip" to true, "storage_port" to true, "pubkey_x25519" to true, "pubkey_ed25519" to true ) ) ) From 18818bf8da16b1ac87bbf491129266bea024fdeb Mon Sep 17 00:00:00 2001 From: jubb Date: Wed, 12 May 2021 11:24:08 +1000 Subject: [PATCH 12/31] refactor: re-add the node limit --- .../src/main/java/org/session/libsession/snode/SnodeAPI.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt index cb0d5ee837..a5094a63ee 100644 --- a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt +++ b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt @@ -92,7 +92,7 @@ object SnodeAPI { "method" to "get_n_service_nodes", "params" to mapOf( "active_only" to true, -// "limit" to 256, + "limit" to 256, "fields" to mapOf( "public_ip" to true, "storage_port" to true, "pubkey_x25519" to true, "pubkey_ed25519" to true ) ) ) From fa5edcefd504987431d9a1fdf0a6818e86d7b97d Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 14:01:57 +1000 Subject: [PATCH 13/31] Minor message type refactoring --- .../messaging/messages/Destination.kt | 16 +-- .../libsession/messaging/messages/Message.kt | 10 +- .../control/ClosedGroupControlMessage.kt | 71 +++++----- .../messages/control/ConfigurationMessage.kt | 49 +++---- .../messages/control/ControlMessage.kt | 3 +- .../control/DataExtractionNotification.kt | 6 +- .../messages/control/ExpirationTimerUpdate.kt | 23 ++- .../messaging/messages/control/ReadReceipt.kt | 14 +- .../messages/control/TypingIndicator.kt | 13 +- .../messaging/messages/visible/LinkPreview.kt | 21 ++- .../messaging/messages/visible/Profile.kt | 17 +-- .../messaging/messages/visible/Quote.kt | 37 +++-- .../messages/visible/VisibleMessage.kt | 134 +++++++++--------- 13 files changed, 204 insertions(+), 210 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/Destination.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/Destination.kt index 4a2d99da84..212e110b25 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/Destination.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/Destination.kt @@ -7,9 +7,6 @@ import org.session.libsession.messaging.threads.Address import org.session.libsession.utilities.GroupUtil import org.session.libsignal.service.loki.utilities.toHexString -typealias OpenGroupModel = OpenGroup -typealias OpenGroupV2Model = OpenGroupV2 - sealed class Destination { class Contact(var publicKey: String) : Destination() { @@ -21,11 +18,12 @@ sealed class Destination { class OpenGroup(var channel: Long, var server: String) : Destination() { internal constructor(): this(0, "") } - class OpenGroupV2(var room: String, var server: String): Destination() { + class OpenGroupV2(var room: String, var server: String) : Destination() { internal constructor(): this("", "") } companion object { + fun from(address: Address): Destination { return when { address.isContact -> { @@ -39,10 +37,12 @@ sealed class Destination { address.isOpenGroup -> { val storage = MessagingModuleConfiguration.shared.storage val threadID = storage.getThreadID(address.contactIdentifier())!! - when (val openGroup = storage.getOpenGroup(threadID) ?: storage.getV2OpenGroup(threadID)) { - is OpenGroupModel -> OpenGroup(openGroup.channel, openGroup.server) - is OpenGroupV2Model -> OpenGroupV2(openGroup.room, openGroup.server) - else -> throw Exception("Invalid OpenGroup $openGroup") + when (val openGroup = storage.getV2OpenGroup(threadID) ?: storage.getOpenGroup(threadID)) { + is org.session.libsession.messaging.open_groups.OpenGroup + -> Destination.OpenGroup(openGroup.channel, openGroup.server) + is org.session.libsession.messaging.open_groups.OpenGroupV2 + -> Destination.OpenGroupV2(openGroup.room, openGroup.server) + else -> throw Exception("Missing open group for thread with ID: $threadID.") } } else -> { diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/Message.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/Message.kt index d6204dc123..323c3fd263 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/Message.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/Message.kt @@ -18,12 +18,10 @@ abstract class Message { open val isSelfSendValid: Boolean = false open fun isValid(): Boolean { - sentTimestamp?.let { - if (it <= 0) return false - } - receivedTimestamp?.let { - if (it <= 0) return false - } + val sentTimestamp = sentTimestamp + if (sentTimestamp != null && sentTimestamp <= 0) { return false } + val receivedTimestamp = receivedTimestamp + if (receivedTimestamp != null && receivedTimestamp <= 0) { return false } return sender != null && recipient != null } diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ClosedGroupControlMessage.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ClosedGroupControlMessage.kt index 78275ca3b2..0d2403cd24 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ClosedGroupControlMessage.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ClosedGroupControlMessage.kt @@ -16,9 +16,10 @@ import org.session.libsignal.utilities.Hex import org.session.libsignal.utilities.logging.Log class ClosedGroupControlMessage() : ControlMessage() { + var kind: Kind? = null - override val ttl: Long = run { - when (kind) { + override val ttl: Long get() { + return when (kind) { is Kind.EncryptionKeyPair -> 14 * 24 * 60 * 60 * 1000 else -> 14 * 24 * 60 * 60 * 1000 } @@ -26,31 +27,46 @@ class ClosedGroupControlMessage() : ControlMessage() { override val isSelfSendValid: Boolean = true - var kind: Kind? = null + override fun isValid(): Boolean { + val kind = kind + if (!super.isValid() || kind == null) return false + return when (kind) { + is Kind.New -> { + !kind.publicKey.isEmpty && kind.name.isNotEmpty() && kind.encryptionKeyPair?.publicKey != null + && kind.encryptionKeyPair?.privateKey != null && kind.members.isNotEmpty() && kind.admins.isNotEmpty() + } + is Kind.EncryptionKeyPair -> true + is Kind.NameChange -> kind.name.isNotEmpty() + is Kind.MembersAdded -> kind.members.isNotEmpty() + is Kind.MembersRemoved -> kind.members.isNotEmpty() + is Kind.MemberLeft -> true + } + } sealed class Kind { class New(var publicKey: ByteString, var name: String, var encryptionKeyPair: ECKeyPair?, var members: List, var admins: List) : Kind() { - internal constructor(): this(ByteString.EMPTY, "", null, listOf(), listOf()) + internal constructor() : this(ByteString.EMPTY, "", null, listOf(), listOf()) } - /// An encryption key pair encrypted for each member individually. - /// - /// - Note: `publicKey` is only set when an encryption key pair is sent in a one-to-one context (i.e. not in a group). + /** An encryption key pair encrypted for each member individually. + * + * **Note:** `publicKey` is only set when an encryption key pair is sent in a one-to-one context (i.e. not in a group). + */ class EncryptionKeyPair(var publicKey: ByteString?, var wrappers: Collection) : Kind() { - internal constructor(): this(null, listOf()) + internal constructor() : this(null, listOf()) } class NameChange(var name: String) : Kind() { - internal constructor(): this("") + internal constructor() : this("") } class MembersAdded(var members: List) : Kind() { - internal constructor(): this(listOf()) + internal constructor() : this(listOf()) } class MembersRemoved(var members: List) : Kind() { - internal constructor(): this(listOf()) + internal constructor() : this(listOf()) } class MemberLeft() : Kind() val description: String = - when(this) { + when (this) { is New -> "new" is EncryptionKeyPair -> "encryptionKeyPair" is NameChange -> "nameChange" @@ -65,18 +81,19 @@ class ClosedGroupControlMessage() : ControlMessage() { fun fromProto(proto: SignalServiceProtos.Content): ClosedGroupControlMessage? { if (!proto.hasDataMessage() || !proto.dataMessage.hasClosedGroupControlMessage()) return null - val closedGroupControlMessageProto = proto.dataMessage?.closedGroupControlMessage!! + val closedGroupControlMessageProto = proto.dataMessage!!.closedGroupControlMessage!! val kind: Kind - when (closedGroupControlMessageProto.type) { + when (closedGroupControlMessageProto.type!!) { DataMessage.ClosedGroupControlMessage.Type.NEW -> { val publicKey = closedGroupControlMessageProto.publicKey ?: return null val name = closedGroupControlMessageProto.name ?: return null val encryptionKeyPairAsProto = closedGroupControlMessageProto.encryptionKeyPair ?: return null try { - val encryptionKeyPair = ECKeyPair(DjbECPublicKey(encryptionKeyPairAsProto.publicKey.toByteArray()), DjbECPrivateKey(encryptionKeyPairAsProto.privateKey.toByteArray())) + val encryptionKeyPair = ECKeyPair(DjbECPublicKey(encryptionKeyPairAsProto.publicKey.toByteArray()), + DjbECPrivateKey(encryptionKeyPairAsProto.privateKey.toByteArray())) kind = Kind.New(publicKey, name, encryptionKeyPair, closedGroupControlMessageProto.membersList, closedGroupControlMessageProto.adminsList) } catch (e: Exception) { - Log.w(TAG, "Couldn't parse key pair") + Log.w(TAG, "Couldn't parse key pair from proto: $encryptionKeyPairAsProto.") return null } } @@ -107,26 +124,10 @@ class ClosedGroupControlMessage() : ControlMessage() { this.kind = kind } - override fun isValid(): Boolean { - if (!super.isValid()) return false - val kind = kind ?: return false - return when(kind) { - is Kind.New -> { - !kind.publicKey.isEmpty && kind.name.isNotEmpty() && kind.encryptionKeyPair!!.publicKey != null - && kind.encryptionKeyPair!!.privateKey != null && kind.members.isNotEmpty() && kind.admins.isNotEmpty() - } - is Kind.EncryptionKeyPair -> true - is Kind.NameChange -> kind.name.isNotEmpty() - is Kind.MembersAdded -> kind.members.isNotEmpty() - is Kind.MembersRemoved -> kind.members.isNotEmpty() - is Kind.MemberLeft -> true - } - } - override fun toProto(): SignalServiceProtos.Content? { val kind = kind if (kind == null) { - Log.w(TAG, "Couldn't construct closed group update proto from: $this") + Log.w(TAG, "Couldn't construct closed group control message proto from: $this.") return null } try { @@ -176,7 +177,7 @@ class ClosedGroupControlMessage() : ControlMessage() { contentProto.dataMessage = dataMessageProto.build() return contentProto.build() } catch (e: Exception) { - Log.w(TAG, "Couldn't construct closed group update proto from: $this") + Log.w(TAG, "Couldn't construct closed group control message proto from: $this.") return null } } @@ -188,6 +189,7 @@ class ClosedGroupControlMessage() : ControlMessage() { } companion object { + fun fromProto(proto: DataMessage.ClosedGroupControlMessage.KeyPairWrapper): KeyPairWrapper { return KeyPairWrapper(proto.publicKey.toByteArray().toHexString(), proto.encryptedKeyPair) } @@ -199,7 +201,6 @@ class ClosedGroupControlMessage() : ControlMessage() { val result = DataMessage.ClosedGroupControlMessage.KeyPairWrapper.newBuilder() result.publicKey = ByteString.copyFrom(Hex.fromStringCondensed(publicKey)) result.encryptedKeyPair = encryptedKeyPair - return try { result.build() } catch (e: Exception) { diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt index 310ec0c019..85f1d8a1b0 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt @@ -14,12 +14,15 @@ import org.session.libsignal.service.loki.utilities.removing05PrefixIfNeeded import org.session.libsignal.service.loki.utilities.toHexString import org.session.libsignal.utilities.Hex -class ConfigurationMessage(var closedGroups: List, var openGroups: List, var contacts: List, var displayName: String, var profilePicture: String?, var profileKey: ByteArray): ControlMessage() { +class ConfigurationMessage(var closedGroups: List, var openGroups: List, var contacts: List, + var displayName: String, var profilePicture: String?, var profileKey: ByteArray) : ControlMessage() { + + override val isSelfSendValid: Boolean = true class ClosedGroup(var publicKey: String, var name: String, var encryptionKeyPair: ECKeyPair?, var members: List, var admins: List) { val isValid: Boolean get() = members.isNotEmpty() && admins.isNotEmpty() - internal constructor(): this("", "", null, listOf(), listOf()) + internal constructor() : this("", "", null, listOf(), listOf()) override fun toString(): String { return name @@ -56,7 +59,7 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: class Contact(var publicKey: String, var name: String, var profilePicture: String?, var profileKey: ByteArray?) { - internal constructor(): this("", "", null, null) + internal constructor() : this("", "", null, null) companion object { @@ -66,8 +69,7 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: val name = proto.name val profilePicture = if (proto.hasProfilePicture()) proto.profilePicture else null val profileKey = if (proto.hasProfileKey()) proto.profileKey.toByteArray() else null - - return Contact(publicKey,name,profilePicture,profileKey) + return Contact(publicKey, name, profilePicture, profileKey) } } @@ -79,18 +81,18 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: } catch (e: Exception) { return null } - if (!this.profilePicture.isNullOrEmpty()) { - result.profilePicture = this.profilePicture + val profilePicture = profilePicture + if (!profilePicture.isNullOrEmpty()) { + result.profilePicture = profilePicture } - if (this.profileKey != null) { - result.profileKey = ByteString.copyFrom(this.profileKey) + val profileKey = profileKey + if (profileKey != null) { + result.profileKey = ByteString.copyFrom(profileKey) } return result.build() } } - override val isSelfSendValid: Boolean = true - companion object { fun getCurrent(contacts: List): ConfigurationMessage? { @@ -103,24 +105,22 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: val profilePicture = TextSecurePreferences.getProfilePictureURL(context) val profileKey = ProfileKeyUtil.getProfileKey(context) val groups = storage.getAllGroups() - for (groupRecord in groups) { - if (groupRecord.isClosedGroup) { - if (!groupRecord.members.contains(Address.fromSerialized(storage.getUserPublicKey()!!))) continue - val groupPublicKey = GroupUtil.doubleDecodeGroupID(groupRecord.encodedId).toHexString() + for (group in groups) { + if (group.isClosedGroup) { + if (!group.members.contains(Address.fromSerialized(storage.getUserPublicKey()!!))) continue + val groupPublicKey = GroupUtil.doubleDecodeGroupID(group.encodedId).toHexString() val encryptionKeyPair = storage.getLatestClosedGroupEncryptionKeyPair(groupPublicKey) ?: continue - val closedGroup = ClosedGroup(groupPublicKey, groupRecord.title, encryptionKeyPair, groupRecord.members.map { it.serialize() }, groupRecord.admins.map { it.serialize() }) + val closedGroup = ClosedGroup(groupPublicKey, group.title, encryptionKeyPair, group.members.map { it.serialize() }, group.admins.map { it.serialize() }) closedGroups.add(closedGroup) } - if (groupRecord.isOpenGroup) { - val threadID = storage.getThreadID(groupRecord.encodedId) ?: continue + if (group.isOpenGroup) { + val threadID = storage.getThreadID(group.encodedId) ?: continue val openGroup = storage.getOpenGroup(threadID) val openGroupV2 = storage.getV2OpenGroup(threadID) - val shareUrl = openGroup?.server ?: openGroupV2?.toJoinUrl() ?: continue openGroups.add(shareUrl) } } - return ConfigurationMessage(closedGroups, openGroups, contacts, displayName, profilePicture, profileKey) } @@ -145,6 +145,7 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: configurationProto.addAllOpenGroups(openGroups) configurationProto.addAllContacts(this.contacts.mapNotNull { it.toProto() }) configurationProto.displayName = displayName + val profilePicture = profilePicture if (!profilePicture.isNullOrEmpty()) { configurationProto.profilePicture = profilePicture } @@ -157,10 +158,10 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: override fun toString(): String { return """ ConfigurationMessage( - closedGroups: ${(closedGroups)} - openGroups: ${(openGroups)} - displayName: $displayName - profilePicture: $profilePicture + closedGroups: ${(closedGroups)}, + openGroups: ${(openGroups)}, + displayName: $displayName, + profilePicture: $profilePicture, profileKey: $profileKey ) """.trimIndent() diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ControlMessage.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ControlMessage.kt index 44cd7ee4d8..fbc013d73e 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ControlMessage.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ControlMessage.kt @@ -2,5 +2,4 @@ package org.session.libsession.messaging.messages.control import org.session.libsession.messaging.messages.Message -abstract class ControlMessage : Message() { -} \ No newline at end of file +abstract class ControlMessage : Message() \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/DataExtractionNotification.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/DataExtractionNotification.kt index 5aec11827b..90cc803713 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/DataExtractionNotification.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/DataExtractionNotification.kt @@ -3,7 +3,7 @@ package org.session.libsession.messaging.messages.control import org.session.libsignal.service.internal.push.SignalServiceProtos import org.session.libsignal.utilities.logging.Log -class DataExtractionNotification(): ControlMessage() { +class DataExtractionNotification() : ControlMessage() { var kind: Kind? = null sealed class Kind { @@ -39,8 +39,8 @@ class DataExtractionNotification(): ControlMessage() { } override fun isValid(): Boolean { - if (!super.isValid()) return false - val kind = kind ?: return false + val kind = kind + if (!super.isValid() || kind == null) return false return when(kind) { is Kind.Screenshot -> true is Kind.MediaSaved -> kind.timestamp > 0 diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt index 5d1854e815..266de5eb92 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt @@ -6,13 +6,20 @@ import org.session.libsignal.utilities.logging.Log import org.session.libsignal.service.internal.push.SignalServiceProtos class ExpirationTimerUpdate() : ControlMessage() { - /// In the case of a sync message, the public key of the person the message was targeted at. - /// - Note: `nil` if this isn't a sync message. + /** In the case of a sync message, the public key of the person the message was targeted at. + * + * **Note:** `nil` if this isn't a sync message. + */ var syncTarget: String? = null var duration: Int? = 0 override val isSelfSendValid: Boolean = true + override fun isValid(): Boolean { + if (!super.isValid()) return false + return duration != null + } + companion object { const val TAG = "ExpirationTimerUpdate" @@ -26,21 +33,11 @@ class ExpirationTimerUpdate() : ControlMessage() { } } - internal constructor(syncTarget: String?, duration: Int) : this() { + internal constructor(syncTarget: String? = null, duration: Int) : this() { this.syncTarget = syncTarget this.duration = duration } - internal constructor(duration: Int) : this() { - this.syncTarget = null - this.duration = duration - } - - override fun isValid(): Boolean { - if (!super.isValid()) return false - return duration != null - } - override fun toProto(): SignalServiceProtos.Content? { val duration = duration if (duration == null) { diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ReadReceipt.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ReadReceipt.kt index a912740da0..1f4bc84e3e 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ReadReceipt.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ReadReceipt.kt @@ -6,6 +6,13 @@ import org.session.libsignal.utilities.logging.Log class ReadReceipt() : ControlMessage() { var timestamps: List? = null + override fun isValid(): Boolean { + if (!super.isValid()) return false + val timestamps = timestamps ?: return false + if (timestamps.isNotEmpty()) { return true } + return false + } + companion object { const val TAG = "ReadReceipt" @@ -22,13 +29,6 @@ class ReadReceipt() : ControlMessage() { this.timestamps = timestamps } - override fun isValid(): Boolean { - if (!super.isValid()) return false - val timestamps = timestamps ?: return false - if (timestamps.isNotEmpty()) { return true } - return false - } - override fun toProto(): SignalServiceProtos.Content? { val timestamps = timestamps if (timestamps == null) { diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/TypingIndicator.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/TypingIndicator.kt index dd26ae7031..a06f821cb8 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/TypingIndicator.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/TypingIndicator.kt @@ -4,9 +4,15 @@ import org.session.libsignal.service.internal.push.SignalServiceProtos import org.session.libsignal.utilities.logging.Log class TypingIndicator() : ControlMessage() { - override val ttl: Long = 30 * 1000 var kind: Kind? = null + override val ttl: Long = 20 * 1000 + + override fun isValid(): Boolean { + if (!super.isValid()) return false + return kind != null + } + companion object { const val TAG = "TypingIndicator" @@ -41,11 +47,6 @@ class TypingIndicator() : ControlMessage() { this.kind = kind } - override fun isValid(): Boolean { - if (!super.isValid()) return false - return kind != null - } - override fun toProto(): SignalServiceProtos.Content? { val timestamp = sentTimestamp val kind = kind diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/LinkPreview.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/LinkPreview.kt index a292bf7c6a..10c41b18b6 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/LinkPreview.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/LinkPreview.kt @@ -10,6 +10,10 @@ class LinkPreview() { var url: String? = null var attachmentID: Long? = 0 + fun isValid(): Boolean { + return (title != null && url != null && attachmentID != null) + } + companion object { const val TAG = "LinkPreview" @@ -20,11 +24,8 @@ class LinkPreview() { } fun from(signalLinkPreview: SignalLinkPreiview?): LinkPreview? { - return if (signalLinkPreview == null) { - null - } else { - LinkPreview(signalLinkPreview.title, signalLinkPreview.url, signalLinkPreview.attachmentId?.rowId) - } + if (signalLinkPreview == null) { return null } + return LinkPreview(signalLinkPreview.title, signalLinkPreview.url, signalLinkPreview.attachmentId?.rowId) } } @@ -34,10 +35,6 @@ class LinkPreview() { this.attachmentID = attachmentID } - fun isValid(): Boolean { - return (title != null && url != null && attachmentID != null) - } - fun toProto(): SignalServiceProtos.DataMessage.Preview? { val url = url if (url == null) { @@ -46,10 +43,10 @@ class LinkPreview() { } val linkPreviewProto = SignalServiceProtos.DataMessage.Preview.newBuilder() linkPreviewProto.url = url - title?.let { linkPreviewProto.title = title } - val attachmentID = attachmentID + title?.let { linkPreviewProto.title = it } + val database = MessagingModuleConfiguration.shared.messageDataProvider attachmentID?.let { - MessagingModuleConfiguration.shared.messageDataProvider.getSignalAttachmentPointer(attachmentID)?.let { + database.getSignalAttachmentPointer(it)?.let { val attachmentProto = Attachment.createAttachmentPointer(it) linkPreviewProto.image = attachmentProto } diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Profile.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Profile.kt index 7464a4be5d..98cb5ecafb 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Profile.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Profile.kt @@ -17,12 +17,11 @@ class Profile() { val displayName = profileProto.displayName ?: return null val profileKey = proto.profileKey val profilePictureURL = profileProto.profilePicture - profileKey?.let { - profilePictureURL?.let { - return Profile(displayName = displayName, profileKey = profileKey.toByteArray(), profilePictureURL = profilePictureURL) - } + if (profileKey != null && profilePictureURL != null) { + return Profile(displayName, profileKey.toByteArray(), profilePictureURL) + } else { + return Profile(displayName) } - return Profile(displayName) } } @@ -35,16 +34,14 @@ class Profile() { fun toProto(): SignalServiceProtos.DataMessage? { val displayName = displayName if (displayName == null) { - Log.w(TAG, "Couldn't construct link preview proto from: $this") + Log.w(TAG, "Couldn't construct profile proto from: $this") return null } val dataMessageProto = SignalServiceProtos.DataMessage.newBuilder() val profileProto = SignalServiceProtos.DataMessage.LokiProfile.newBuilder() profileProto.displayName = displayName - val profileKey = profileKey - profileKey?.let { dataMessageProto.profileKey = ByteString.copyFrom(profileKey) } - val profilePictureURL = profilePictureURL - profilePictureURL?.let { profileProto.profilePicture = profilePictureURL } + profileKey?.let { dataMessageProto.profileKey = ByteString.copyFrom(it) } + profilePictureURL?.let { profileProto.profilePicture = it } // Build try { dataMessageProto.profile = profileProto.build() diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Quote.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Quote.kt index 88bf089a1c..376f52fd25 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Quote.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/Quote.kt @@ -13,6 +13,10 @@ class Quote() { var text: String? = null var attachmentID: Long? = null + fun isValid(): Boolean { + return (timestamp != null && publicKey != null) + } + companion object { const val TAG = "Quote" @@ -24,12 +28,9 @@ class Quote() { } fun from(signalQuote: SignalQuote?): Quote? { - return if (signalQuote == null) { - null - } else { - val attachmentID = (signalQuote.attachments?.firstOrNull() as? DatabaseAttachment)?.attachmentId?.rowId - Quote(signalQuote.id, signalQuote.author.serialize(), signalQuote.text, attachmentID) - } + if (signalQuote == null) { return null } + val attachmentID = (signalQuote.attachments?.firstOrNull() as? DatabaseAttachment)?.attachmentId?.rowId + return Quote(signalQuote.id, signalQuote.author.serialize(), signalQuote.text, attachmentID) } } @@ -40,10 +41,6 @@ class Quote() { this.attachmentID = attachmentID } - fun isValid(): Boolean { - return (timestamp != null && publicKey != null) - } - fun toProto(): SignalServiceProtos.DataMessage.Quote? { val timestamp = timestamp val publicKey = publicKey @@ -54,7 +51,7 @@ class Quote() { val quoteProto = SignalServiceProtos.DataMessage.Quote.newBuilder() quoteProto.id = timestamp quoteProto.author = publicKey - text?.let { quoteProto.text = text } + text?.let { quoteProto.text = it } addAttachmentsIfNeeded(quoteProto) // Build try { @@ -66,23 +63,23 @@ class Quote() { } private fun addAttachmentsIfNeeded(quoteProto: SignalServiceProtos.DataMessage.Quote.Builder) { - if (attachmentID == null) return - val attachment = MessagingModuleConfiguration.shared.messageDataProvider.getSignalAttachmentPointer(attachmentID!!) - if (attachment == null) { + val attachmentID = attachmentID ?: return + val database = MessagingModuleConfiguration.shared.messageDataProvider + val pointer = database.getSignalAttachmentPointer(attachmentID) + if (pointer == null) { Log.w(TAG, "Ignoring invalid attachment for quoted message.") return } - if (attachment.url.isNullOrEmpty()) { + if (pointer.url.isNullOrEmpty()) { if (BuildConfig.DEBUG) { - //TODO equivalent to iOS's preconditionFailure - Log.d(TAG,"Sending a message before all associated attachments have been uploaded.") + Log.w(TAG,"Sending a message before all associated attachments have been uploaded.") return } } val quotedAttachmentProto = SignalServiceProtos.DataMessage.Quote.QuotedAttachment.newBuilder() - quotedAttachmentProto.contentType = attachment.contentType - if (attachment.fileName.isPresent) quotedAttachmentProto.fileName = attachment.fileName.get() - quotedAttachmentProto.thumbnail = Attachment.createAttachmentPointer(attachment) + quotedAttachmentProto.contentType = pointer.contentType + if (pointer.fileName.isPresent) { quotedAttachmentProto.fileName = pointer.fileName.get() } + quotedAttachmentProto.thumbnail = Attachment.createAttachmentPointer(pointer) try { quoteProto.addAttachments(quotedAttachmentProto.build()) } catch (e: Exception) { diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/VisibleMessage.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/VisibleMessage.kt index 63756c0948..8c795d22e8 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/visible/VisibleMessage.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/visible/VisibleMessage.kt @@ -12,6 +12,10 @@ import org.session.libsignal.utilities.logging.Log import org.session.libsession.messaging.sending_receiving.attachments.Attachment as SignalAttachment class VisibleMessage : Message() { + /** In the case of a sync message, the public key of the person the message was targeted at. + * + * **Note:** `nil` if this isn't a sync message. + */ var syncTarget: String? = null var text: String? = null val attachmentIDs: MutableList = mutableListOf() @@ -21,46 +25,7 @@ class VisibleMessage : Message() { override val isSelfSendValid: Boolean = true - companion object { - const val TAG = "VisibleMessage" - - fun fromProto(proto: SignalServiceProtos.Content): VisibleMessage? { - val dataMessage = if (proto.hasDataMessage()) proto.dataMessage else return null - val result = VisibleMessage() - if (dataMessage.hasSyncTarget()) { - result.syncTarget = dataMessage.syncTarget - } - result.text = dataMessage.body - // Attachments are handled in MessageReceiver - val quoteProto = if (dataMessage.hasQuote()) dataMessage.quote else null - quoteProto?.let { - val quote = Quote.fromProto(quoteProto) - quote?.let { result.quote = quote } - } - val linkPreviewProto = dataMessage.previewList.firstOrNull() - linkPreviewProto?.let { - val linkPreview = LinkPreview.fromProto(linkPreviewProto) - linkPreview?.let { result.linkPreview = linkPreview } - } - // TODO Contact - val profile = Profile.fromProto(dataMessage) - profile?.let { result.profile = profile } - return result - } - } - - fun addSignalAttachments(signalAttachments: List) { - val attachmentIDs = signalAttachments.map { - val databaseAttachment = it as DatabaseAttachment - databaseAttachment.attachmentId.rowId - } - this.attachmentIDs.addAll(attachmentIDs) - } - - fun isMediaMessage(): Boolean { - return attachmentIDs.isNotEmpty() || quote != null || linkPreview != null - } - + // region Validation override fun isValid(): Boolean { if (!super.isValid()) return false if (attachmentIDs.isNotEmpty()) return true @@ -68,56 +33,84 @@ class VisibleMessage : Message() { if (text.isNotEmpty()) return true return false } + // endregion + + // region Proto Conversion + companion object { + const val TAG = "VisibleMessage" + + fun fromProto(proto: SignalServiceProtos.Content): VisibleMessage? { + val dataMessage = proto.dataMessage ?: return null + val result = VisibleMessage() + if (dataMessage.hasSyncTarget()) { result.syncTarget = dataMessage.syncTarget } + result.text = dataMessage.body + // Attachments are handled in MessageReceiver + val quoteProto = if (dataMessage.hasQuote()) dataMessage.quote else null + if (quoteProto != null) { + val quote = Quote.fromProto(quoteProto) + result.quote = quote + } + val linkPreviewProto = dataMessage.previewList.firstOrNull() + if (linkPreviewProto != null) { + val linkPreview = LinkPreview.fromProto(linkPreviewProto) + result.linkPreview = linkPreview + } + // TODO: Contact + val profile = Profile.fromProto(dataMessage) + if (profile != null) { result.profile = profile } + return result + } + } override fun toProto(): SignalServiceProtos.Content? { val proto = SignalServiceProtos.Content.newBuilder() val dataMessage: SignalServiceProtos.DataMessage.Builder // Profile - val profile = profile - val profileProto = profile?.toProto() + val profileProto = profile?.let { it.toProto() } if (profileProto != null) { dataMessage = profileProto.toBuilder() } else { dataMessage = SignalServiceProtos.DataMessage.newBuilder() } // Text - text?.let { dataMessage.body = text } + if (text != null) { dataMessage.body = text } // Quote - quote?.let { - val quoteProto = it.toProto() - if (quoteProto != null) dataMessage.quote = quoteProto + val quoteProto = quote?.let { it.toProto() } + if (quoteProto != null) { + dataMessage.quote = quoteProto } - //Link preview - linkPreview?.let { - val linkPreviewProto = it.toProto() - linkPreviewProto?.let { - dataMessage.addAllPreview(listOf(linkPreviewProto)) - } + // Link preview + val linkPreviewProto = linkPreview?.let { it.toProto() } + if (linkPreviewProto != null) { + dataMessage.addAllPreview(listOf(linkPreviewProto)) } - //Attachments - val attachments = attachmentIDs.mapNotNull { MessagingModuleConfiguration.shared.messageDataProvider.getSignalAttachmentPointer(it) } - if (!attachments.all { !it.url.isNullOrEmpty() }) { + // Attachments + val database = MessagingModuleConfiguration.shared.messageDataProvider + val attachments = attachmentIDs.mapNotNull { database.getSignalAttachmentPointer(it) } + if (attachments.any { it.url.isNullOrEmpty() }) { if (BuildConfig.DEBUG) { - //TODO equivalent to iOS's preconditionFailure - Log.d(TAG, "Sending a message before all associated attachments have been uploaded.") + Log.w(TAG, "Sending a message before all associated attachments have been uploaded.") } } - val attachmentPointers = attachments.mapNotNull { Attachment.createAttachmentPointer(it) } - dataMessage.addAllAttachments(attachmentPointers) - // TODO Contact + val pointers = attachments.mapNotNull { Attachment.createAttachmentPointer(it) } + dataMessage.addAllAttachments(pointers) + // TODO: Contact // Expiration timer // TODO: We * want * expiration timer updates to be explicit. But currently Android will disable the expiration timer for a conversation - // if it receives a message without the current expiration timer value attached to it... + // if it receives a message without the current expiration timer value attached to it... val storage = MessagingModuleConfiguration.shared.storage val context = MessagingModuleConfiguration.shared.context - val expiration = if (storage.isClosedGroup(recipient!!)) Recipient.from(context, Address.fromSerialized(GroupUtil.doubleEncodeGroupID(recipient!!)), false).expireMessages - else Recipient.from(context, Address.fromSerialized(recipient!!), false).expireMessages + val expiration = if (storage.isClosedGroup(recipient!!)) { + Recipient.from(context, Address.fromSerialized(GroupUtil.doubleEncodeGroupID(recipient!!)), false).expireMessages + } else { + Recipient.from(context, Address.fromSerialized(recipient!!), false).expireMessages + } dataMessage.expireTimer = expiration // Group context if (storage.isClosedGroup(recipient!!)) { try { setGroupContext(dataMessage) - } catch(e: Exception) { + } catch (e: Exception) { Log.w(TAG, "Couldn't construct visible message proto from: $this") return null } @@ -135,4 +128,17 @@ class VisibleMessage : Message() { return null } } + // endregion + + fun addSignalAttachments(signalAttachments: List) { + val attachmentIDs = signalAttachments.map { + val databaseAttachment = it as DatabaseAttachment + databaseAttachment.attachmentId.rowId + } + this.attachmentIDs.addAll(attachmentIDs) + } + + fun isMediaMessage(): Boolean { + return attachmentIDs.isNotEmpty() || quote != null || linkPreview != null + } } \ No newline at end of file From 21698fcba51625137ca25d5622e8e6f57d654002 Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 14:02:07 +1000 Subject: [PATCH 14/31] Update version number --- app/build.gradle | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/build.gradle b/app/build.gradle index 2888a2ba07..74738b8801 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -158,8 +158,8 @@ dependencies { testImplementation 'org.robolectric:shadows-multidex:4.2' } -def canonicalVersionCode = 158 -def canonicalVersionName = "1.10.1" +def canonicalVersionCode = 159 +def canonicalVersionName = "1.10.2" def postFixSize = 10 def abiPostFix = ['armeabi-v7a' : 1, From d8932416f145696607bed624223cf8d2f0201bac Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 14:48:13 +1000 Subject: [PATCH 15/31] Minor V2 open group refactoring --- .../securesms/database/Storage.kt | 2 +- .../loki/activities/JoinPublicChatActivity.kt | 2 +- .../loki/database/LokiThreadDatabase.kt | 4 +- .../messaging/file_server/FileServerAPIV2.kt | 7 +- .../messages/control/ConfigurationMessage.kt | 2 +- .../messages/control/ExpirationTimerUpdate.kt | 7 +- .../messaging/open_groups/OpenGroupAPIV2.kt | 292 ++++++++---------- .../open_groups/OpenGroupMessageV2.kt | 54 ++-- .../messaging/open_groups/OpenGroupV2.kt | 41 ++- .../ReceivedMessageHandler.kt | 2 +- 10 files changed, 197 insertions(+), 216 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt index 0ab4824343..6a2ff6e8df 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt @@ -257,7 +257,7 @@ class Storage(context: Context, helper: SQLCipherOpenHelper) : Database(context, val database = databaseHelper.readableDatabase return database.get(LokiThreadDatabase.publicChatTable, "${LokiThreadDatabase.threadID} = ?", arrayOf(threadId)) { cursor -> val publicChatAsJson = cursor.getString(LokiThreadDatabase.publicChat) - OpenGroupV2.fromJson(publicChatAsJson) + OpenGroupV2.fromJSON(publicChatAsJson) } } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/activities/JoinPublicChatActivity.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/activities/JoinPublicChatActivity.kt index 6686ca8345..f0490d379d 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/activities/JoinPublicChatActivity.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/activities/JoinPublicChatActivity.kt @@ -195,7 +195,7 @@ class EnterChatURLFragment : Fragment() { chip.chipIcon = drawable chip.text = defaultGroup.name chip.setOnClickListener { - (requireActivity() as JoinPublicChatActivity).joinPublicChatIfPossible(defaultGroup.toJoinUrl()) + (requireActivity() as JoinPublicChatActivity).joinPublicChatIfPossible(defaultGroup.joinURL) } defaultRoomsGridLayout.addView(chip) } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiThreadDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiThreadDatabase.kt index ba9c0ff477..68ca31cea4 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiThreadDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/LokiThreadDatabase.kt @@ -68,7 +68,7 @@ class LokiThreadDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa while (cursor != null && cursor.moveToNext()) { val threadID = cursor.getLong(threadID) val string = cursor.getString(publicChat) - val openGroup = OpenGroupV2.fromJson(string) + val openGroup = OpenGroupV2.fromJSON(string) if (openGroup != null) result[threadID] = openGroup } } catch (e: Exception) { @@ -100,7 +100,7 @@ class LokiThreadDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa val database = databaseHelper.readableDatabase return database.get(publicChatTable, "${Companion.threadID} = ?", arrayOf(threadID.toString())) { cursor -> val json = cursor.getString(publicChat) - OpenGroupV2.fromJson(json) + OpenGroupV2.fromJSON(json) } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt b/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt index 9469514871..1762a797d9 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt @@ -1,17 +1,14 @@ package org.session.libsession.messaging.file_server import nl.komponents.kovenant.Promise -import nl.komponents.kovenant.functional.bind import nl.komponents.kovenant.functional.map import okhttp3.Headers import okhttp3.HttpUrl import okhttp3.MediaType import okhttp3.RequestBody -import org.session.libsession.messaging.MessagingModuleConfiguration import org.session.libsession.messaging.open_groups.OpenGroupAPIV2 import org.session.libsession.snode.OnionRequestAPI import org.session.libsignal.service.loki.HTTP -import org.session.libsignal.service.loki.utilities.retryIfNeeded import org.session.libsignal.utilities.Base64 import org.session.libsignal.utilities.JsonUtil import org.session.libsignal.utilities.logging.Log @@ -51,7 +48,7 @@ object FileServerAPIV2 { } private fun send(request: Request): Promise, Exception> { - val parsed = HttpUrl.parse(DEFAULT_SERVER) ?: return Promise.ofFail(OpenGroupAPIV2.Error.INVALID_URL) + val parsed = HttpUrl.parse(DEFAULT_SERVER) ?: return Promise.ofFail(OpenGroupAPIV2.Error.InvalidURL) val urlBuilder = HttpUrl.Builder() .scheme(parsed.scheme()) .host(parsed.host()) @@ -91,7 +88,7 @@ object FileServerAPIV2 { val parameters = mapOf("file" to base64EncodedFile) val request = Request(verb = HTTP.Verb.POST, endpoint = "files", parameters = parameters) return send(request).map { json -> - json["result"] as? Long ?: throw OpenGroupAPIV2.Error.PARSING_FAILED + json["result"] as? Long ?: throw OpenGroupAPIV2.Error.ParsingFailed } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt index 85f1d8a1b0..1a15d34860 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ConfigurationMessage.kt @@ -117,7 +117,7 @@ class ConfigurationMessage(var closedGroups: List, var openGroups: val threadID = storage.getThreadID(group.encodedId) ?: continue val openGroup = storage.getOpenGroup(threadID) val openGroupV2 = storage.getV2OpenGroup(threadID) - val shareUrl = openGroup?.server ?: openGroupV2?.toJoinUrl() ?: continue + val shareUrl = openGroup?.server ?: openGroupV2?.joinURL ?: continue openGroups.add(shareUrl) } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt index 266de5eb92..9aa777782a 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/messages/control/ExpirationTimerUpdate.kt @@ -33,7 +33,12 @@ class ExpirationTimerUpdate() : ControlMessage() { } } - internal constructor(syncTarget: String? = null, duration: Int) : this() { + internal constructor(duration: Int) : this() { + this.syncTarget = null + this.duration = duration + } + + internal constructor(syncTarget: String, duration: Int) : this() { this.syncTarget = syncTarget this.duration = duration } diff --git a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt index 7714a66e5d..7f88cd3bc8 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt @@ -5,7 +5,6 @@ import com.fasterxml.jackson.databind.annotation.JsonNaming import com.fasterxml.jackson.databind.type.TypeFactory import kotlinx.coroutines.* import kotlinx.coroutines.flow.MutableSharedFlow -import nl.komponents.kovenant.Kovenant import nl.komponents.kovenant.Promise import nl.komponents.kovenant.functional.bind import nl.komponents.kovenant.functional.map @@ -14,7 +13,6 @@ import okhttp3.HttpUrl import okhttp3.MediaType import okhttp3.RequestBody import org.session.libsession.messaging.MessagingModuleConfiguration -import org.session.libsession.messaging.open_groups.OpenGroupAPIV2.Error import org.session.libsession.snode.OnionRequestAPI import org.session.libsession.utilities.AESGCM import org.session.libsignal.service.loki.HTTP @@ -29,62 +27,48 @@ import org.whispersystems.curve25519.Curve25519 import java.util.* object OpenGroupAPIV2 { - private val moderators: HashMap> = hashMapOf() // Server URL to (channel ID to set of moderator IDs) - const val DEFAULT_SERVER = "http://116.203.70.33" - private const val DEFAULT_SERVER_PUBLIC_KEY = "a03c383cf63c3c4efe67acc52112a6dd734b3a946b9545f488aaa93da7991238" - + private val curve = Curve25519.getInstance(Curve25519.BEST) val defaultRooms = MutableSharedFlow>(replay = 1) - private val curve = Curve25519.getInstance(Curve25519.BEST) + private const val DEFAULT_SERVER_PUBLIC_KEY = "a03c383cf63c3c4efe67acc52112a6dd734b3a946b9545f488aaa93da7991238" + const val DEFAULT_SERVER = "http://116.203.70.33" sealed class Error : Exception() { - object GENERIC : Error() - object PARSING_FAILED : Error() - object DECRYPTION_FAILED : Error() - object SIGNING_FAILED : Error() - object INVALID_URL : Error() - object NO_PUBLIC_KEY : Error() + object Generic : Error() + object ParsingFailed : Error() + object DecryptionFailed : Error() + object SigningFailed : Error() + object InvalidURL : Error() + object NoPublicKey : Error() fun errorDescription() = when (this) { - Error.GENERIC -> "An error occurred." - Error.PARSING_FAILED -> "Invalid response." - Error.DECRYPTION_FAILED -> "Couldn't decrypt response." - Error.SIGNING_FAILED -> "Couldn't sign message." - Error.INVALID_URL -> "Invalid URL." - Error.NO_PUBLIC_KEY -> "Couldn't find server public key." + Error.Generic -> "An error occurred." + Error.ParsingFailed -> "Invalid response." + Error.DecryptionFailed -> "Couldn't decrypt response." + Error.SigningFailed -> "Couldn't sign message." + Error.InvalidURL -> "Invalid URL." + Error.NoPublicKey -> "Couldn't find server public key." } } - data class DefaultGroup(val id: String, - val name: String, - val image: ByteArray?) { - fun toJoinUrl(): String = "$DEFAULT_SERVER/$id?public_key=$DEFAULT_SERVER_PUBLIC_KEY" + data class DefaultGroup(val id: String, val name: String, val image: ByteArray?) { + + val joinURL: String get() = "$DEFAULT_SERVER/$id?public_key=$DEFAULT_SERVER_PUBLIC_KEY" } - data class Info( - val id: String, - val name: String, - val imageID: String? - ) + data class Info(val id: String, val name: String, val imageID: String?) @JsonNaming(PropertyNamingStrategy.SnakeCaseStrategy::class) - data class CompactPollRequest(val roomId: String, - val authToken: String, - val fromDeletionServerId: Long?, - val fromMessageServerId: Long? - ) - - data class CompactPollResult(val messages: List, - val deletions: List, - val moderators: List - ) + data class CompactPollRequest(val roomID: String, val authToken: String, val fromDeletionServerID: Long?, val fromMessageServerID: Long?) + data class CompactPollResult(val messages: List, val deletions: List, val moderators: List) @JsonNaming(PropertyNamingStrategy.SnakeCaseStrategy::class) - data class MessageDeletion @JvmOverloads constructor(val id: Long = 0, - val deletedMessageId: Long = 0 + data class MessageDeletion + @JvmOverloads constructor(val id: Long = 0, val deletedMessageId: Long = 0 ) { + companion object { val EMPTY = MessageDeletion() } @@ -99,38 +83,37 @@ object OpenGroupAPIV2 { val parameters: Any? = null, val headers: Map = mapOf(), val isAuthRequired: Boolean = true, - // Always `true` under normal circumstances. You might want to disable - // this when running over Lokinet. + /** + * Always `true` under normal circumstances. You might want to disable + * this when running over Lokinet. + */ val useOnionRouting: Boolean = true ) private fun createBody(parameters: Any?): RequestBody? { if (parameters == null) return null - val parametersAsJSON = JsonUtil.toJson(parameters) return RequestBody.create(MediaType.get("application/json"), parametersAsJSON) } private fun send(request: Request, isJsonRequired: Boolean = true): Promise, Exception> { - val parsed = HttpUrl.parse(request.server) ?: return Promise.ofFail(Error.INVALID_URL) + val url = HttpUrl.parse(request.server) ?: return Promise.ofFail(Error.InvalidURL) val urlBuilder = HttpUrl.Builder() - .scheme(parsed.scheme()) - .host(parsed.host()) - .port(parsed.port()) - .addPathSegments(request.endpoint) - + .scheme(url.scheme()) + .host(url.host()) + .port(url.port()) + .addPathSegments(request.endpoint) if (request.verb == GET) { for ((key, value) in request.queryParameters) { urlBuilder.addQueryParameter(key, value) } } - fun execute(token: String?): Promise, Exception> { val requestBuilder = okhttp3.Request.Builder() - .url(urlBuilder.build()) - .headers(Headers.of(request.headers)) + .url(urlBuilder.build()) + .headers(Headers.of(request.headers)) if (request.isAuthRequired) { - if (token.isNullOrEmpty()) throw IllegalStateException("No auth token for request") + if (token.isNullOrEmpty()) throw IllegalStateException("No auth token for request.") requestBuilder.header("Authorization", token) } when (request.verb) { @@ -139,25 +122,25 @@ object OpenGroupAPIV2 { POST -> requestBuilder.post(createBody(request.parameters)!!) DELETE -> requestBuilder.delete(createBody(request.parameters)) } - if (!request.room.isNullOrEmpty()) { requestBuilder.header("Room", request.room) } - if (request.useOnionRouting) { val publicKey = MessagingModuleConfiguration.shared.storage.getOpenGroupPublicKey(request.server) - ?: return Promise.ofFail(Error.NO_PUBLIC_KEY) - return OnionRequestAPI.sendOnionRequest(requestBuilder.build(), request.server, publicKey, isJSONRequired = isJsonRequired) - .fail { e -> - if (e is OnionRequestAPI.HTTPRequestFailedAtDestinationException && e.statusCode == 401) { - val storage = MessagingModuleConfiguration.shared.storage - if (request.room != null) { - storage.removeAuthToken("${request.server}.${request.room}") - } else { - storage.removeAuthToken(request.server) - } - } + ?: return Promise.ofFail(Error.NoPublicKey) + return OnionRequestAPI.sendOnionRequest(requestBuilder.build(), request.server, publicKey, isJSONRequired = isJsonRequired).fail { e -> + // A 401 means that we didn't provide a (valid) auth token for a route that required one. We use this as an + // indication that the token we're using has expired. Note that a 403 has a different meaning; it means that + // we provided a valid token but it doesn't have a high enough permission level for the route in question. + if (e is OnionRequestAPI.HTTPRequestFailedAtDestinationException && e.statusCode == 401) { + val storage = MessagingModuleConfiguration.shared.storage + if (request.room != null) { + storage.removeAuthToken("${request.server}.${request.room}") + } else { + storage.removeAuthToken(request.server) } + } + } } else { return Promise.ofFail(IllegalStateException("It's currently not allowed to send non onion routed requests.")) } @@ -172,52 +155,51 @@ object OpenGroupAPIV2 { fun downloadOpenGroupProfilePicture(roomID: String, server: String): Promise { val request = Request(verb = GET, room = roomID, server = server, endpoint = "rooms/$roomID/image", isAuthRequired = false) return send(request).map { json -> - val result = json["result"] as? String ?: throw Error.PARSING_FAILED + val result = json["result"] as? String ?: throw Error.ParsingFailed decode(result) } } + // region Authorization fun getAuthToken(room: String, server: String): Promise { val storage = MessagingModuleConfiguration.shared.storage return storage.getAuthToken(room, server)?.let { Promise.of(it) } ?: run { requestNewAuthToken(room, server) - .bind { claimAuthToken(it, room, server) } - .success { authToken -> - storage.setAuthToken(room, server, authToken) - } + .bind { claimAuthToken(it, room, server) } + .success { authToken -> + storage.setAuthToken(room, server, authToken) + } } } fun requestNewAuthToken(room: String, server: String): Promise { val (publicKey, privateKey) = MessagingModuleConfiguration.shared.storage.getUserKeyPair() - ?: return Promise.ofFail(Error.GENERIC) - val queryParameters = mutableMapOf("public_key" to publicKey) + ?: return Promise.ofFail(Error.Generic) + val queryParameters = mutableMapOf( "public_key" to publicKey ) val request = Request(GET, room, server, "auth_token_challenge", queryParameters, isAuthRequired = false, parameters = null) return send(request).map { json -> - val challenge = json["challenge"] as? Map<*, *> ?: throw Error.PARSING_FAILED - val base64EncodedCiphertext = challenge["ciphertext"] as? String - ?: throw Error.PARSING_FAILED - val base64EncodedEphemeralPublicKey = challenge["ephemeral_public_key"] as? String - ?: throw Error.PARSING_FAILED + val challenge = json["challenge"] as? Map<*, *> ?: throw Error.ParsingFailed + val base64EncodedCiphertext = challenge["ciphertext"] as? String ?: throw Error.ParsingFailed + val base64EncodedEphemeralPublicKey = challenge["ephemeral_public_key"] as? String ?: throw Error.ParsingFailed val ciphertext = decode(base64EncodedCiphertext) val ephemeralPublicKey = decode(base64EncodedEphemeralPublicKey) val symmetricKey = AESGCM.generateSymmetricKey(ephemeralPublicKey, privateKey) val tokenAsData = try { AESGCM.decrypt(ciphertext, symmetricKey) } catch (e: Exception) { - throw Error.DECRYPTION_FAILED + throw Error.DecryptionFailed } tokenAsData.toHexString() } } fun claimAuthToken(authToken: String, room: String, server: String): Promise { - val parameters = mapOf("public_key" to MessagingModuleConfiguration.shared.storage.getUserPublicKey()!!) - val headers = mapOf("Authorization" to authToken) + val parameters = mapOf( "public_key" to MessagingModuleConfiguration.shared.storage.getUserPublicKey()!! ) + val headers = mapOf( "Authorization" to authToken ) val request = Request(verb = POST, room = room, server = server, endpoint = "claim_auth_token", - parameters = parameters, headers = headers, isAuthRequired = false) + parameters = parameters, headers = headers, isAuthRequired = false) return send(request).map { authToken } } @@ -227,33 +209,36 @@ object OpenGroupAPIV2 { MessagingModuleConfiguration.shared.storage.removeAuthToken(room, server) } } + // endregion - // region Sending + // region Upload/Download fun upload(file: ByteArray, room: String, server: String): Promise { val base64EncodedFile = encodeBytes(file) - val parameters = mapOf("file" to base64EncodedFile) + val parameters = mapOf( "file" to base64EncodedFile ) val request = Request(verb = POST, room = room, server = server, endpoint = "files", parameters = parameters) return send(request).map { json -> - json["result"] as? Long ?: throw Error.PARSING_FAILED + json["result"] as? Long ?: throw Error.ParsingFailed } } fun download(file: Long, room: String, server: String): Promise { val request = Request(verb = GET, room = room, server = server, endpoint = "files/$file") return send(request).map { json -> - val base64EncodedFile = json["result"] as? String ?: throw Error.PARSING_FAILED - decode(base64EncodedFile) ?: throw Error.PARSING_FAILED + val base64EncodedFile = json["result"] as? String ?: throw Error.ParsingFailed + decode(base64EncodedFile) ?: throw Error.ParsingFailed } } + // endregion + // region Sending fun send(message: OpenGroupMessageV2, room: String, server: String): Promise { - val signedMessage = message.sign() ?: return Promise.ofFail(Error.SIGNING_FAILED) + val signedMessage = message.sign() ?: return Promise.ofFail(Error.SigningFailed) val jsonMessage = signedMessage.toJSON() val request = Request(verb = POST, room = room, server = server, endpoint = "messages", parameters = jsonMessage) return send(request).map { json -> @Suppress("UNCHECKED_CAST") val rawMessage = json["message"] as? Map - ?: throw Error.PARSING_FAILED - OpenGroupMessageV2.fromJSON(rawMessage) ?: throw Error.PARSING_FAILED + ?: throw Error.ParsingFailed + OpenGroupMessageV2.fromJSON(rawMessage) ?: throw Error.ParsingFailed } } // endregion @@ -268,10 +253,9 @@ object OpenGroupAPIV2 { val request = Request(verb = GET, room = room, server = server, endpoint = "messages", queryParameters = queryParameters) return send(request).map { jsonList -> @Suppress("UNCHECKED_CAST") val rawMessages = jsonList["messages"] as? List> - ?: throw Error.PARSING_FAILED - val lastMessageServerId = storage.getLastMessageServerId(room, server) ?: 0 - - var currentMax = lastMessageServerId + ?: throw Error.ParsingFailed + val lastMessageServerID = storage.getLastMessageServerId(room, server) ?: 0 + var currentLastMessageServerID = lastMessageServerID val messages = rawMessages.mapNotNull { json -> try { val message = OpenGroupMessageV2.fromJSON(json) ?: return@mapNotNull null @@ -285,15 +269,15 @@ object OpenGroupAPIV2 { Log.d("Loki", "Ignoring message with invalid signature") return@mapNotNull null } - if (message.serverID > lastMessageServerId) { - currentMax = message.serverID + if (message.serverID > lastMessageServerID) { + currentLastMessageServerID = message.serverID } message } catch (e: Exception) { null } } - storage.setLastMessageServerId(room, server, currentMax) + storage.setLastMessageServerId(room, server, currentLastMessageServerID) messages } } @@ -304,7 +288,7 @@ object OpenGroupAPIV2 { fun deleteMessage(serverID: Long, room: String, server: String): Promise { val request = Request(verb = DELETE, room = room, server = server, endpoint = "messages/$serverID") return send(request).map { - Log.d("Loki", "Deleted server message") + Log.d("Loki", "Message deletion successful.") } } @@ -318,7 +302,7 @@ object OpenGroupAPIV2 { return send(request).map { json -> val type = TypeFactory.defaultInstance().constructCollectionType(List::class.java, MessageDeletion::class.java) val idsAsString = JsonUtil.toJson(json["ids"]) - val serverIDs = JsonUtil.fromJson>(idsAsString, type) ?: throw Error.PARSING_FAILED + val serverIDs = JsonUtil.fromJson>(idsAsString, type) ?: throw Error.ParsingFailed val lastMessageServerId = storage.getLastDeletionServerId(room, server) ?: 0 val serverID = serverIDs.maxByOrNull {it.id } ?: MessageDeletion.EMPTY if (serverID.id > lastMessageServerId) { @@ -338,7 +322,7 @@ object OpenGroupAPIV2 { val request = Request(verb = GET, room = room, server = server, endpoint = "moderators") return send(request).map { json -> @Suppress("UNCHECKED_CAST") val moderatorsJson = json["moderators"] as? List - ?: throw Error.PARSING_FAILED + ?: throw Error.ParsingFailed val id = "$server.$room" handleModerators(id, moderatorsJson) moderatorsJson @@ -347,90 +331,86 @@ object OpenGroupAPIV2 { @JvmStatic fun ban(publicKey: String, room: String, server: String): Promise { - val parameters = mapOf("public_key" to publicKey) + val parameters = mapOf( "public_key" to publicKey ) val request = Request(verb = POST, room = room, server = server, endpoint = "block_list", parameters = parameters) return send(request).map { - Log.d("Loki", "Banned user $publicKey from $server.$room") + Log.d("Loki", "Banned user: $publicKey from: $server.$room.") } } fun unban(publicKey: String, room: String, server: String): Promise { val request = Request(verb = DELETE, room = room, server = server, endpoint = "block_list/$publicKey") return send(request).map { - Log.d("Loki", "Unbanned user $publicKey from $server.$room") + Log.d("Loki", "Unbanned user: $publicKey from: $server.$room") } } @JvmStatic fun isUserModerator(publicKey: String, room: String, server: String): Boolean = - moderators["$server.$room"]?.contains(publicKey) ?: false + moderators["$server.$room"]?.contains(publicKey) ?: false // endregion // region General @Suppress("UNCHECKED_CAST") fun getCompactPoll(rooms: List, server: String): Promise, Exception> { - val requestAuths = rooms.associateWith { room -> getAuthToken(room, server) } + val authTokenRequests = rooms.associateWith { room -> getAuthToken(room, server) } val storage = MessagingModuleConfiguration.shared.storage val requests = rooms.mapNotNull { room -> val authToken = try { - requestAuths[room]?.get() + authTokenRequests[room]?.get() } catch (e: Exception) { - Log.e("Loki", "Failed to get auth token for $room", e) + Log.e("Loki", "Failed to get auth token for $room.", e) null } ?: return@mapNotNull null - - CompactPollRequest(roomId = room, - authToken = authToken, - fromDeletionServerId = storage.getLastDeletionServerId(room, server), - fromMessageServerId = storage.getLastMessageServerId(room, server) + CompactPollRequest( + roomID = room, + authToken = authToken, + fromDeletionServerID = storage.getLastDeletionServerId(room, server), + fromMessageServerID = storage.getLastMessageServerId(room, server) ) } - val request = Request(verb = POST, room = null, server = server, endpoint = "compact_poll", isAuthRequired = false, parameters = mapOf("requests" to requests)) - // build a request for all rooms + val request = Request(verb = POST, room = null, server = server, endpoint = "compact_poll", isAuthRequired = false, parameters = mapOf( "requests" to requests )) return send(request = request).map { json -> - val results = json["results"] as? List<*> ?: throw Error.PARSING_FAILED - - results.mapNotNull { roomJson -> - if (roomJson !is Map<*,*>) return@mapNotNull null - val roomId = roomJson["room_id"] as? String ?: return@mapNotNull null - - // check the status was fine - val statusCode = roomJson["status_code"] as? Int ?: return@mapNotNull null + val results = json["results"] as? List<*> ?: throw Error.ParsingFailed + results.mapNotNull { json -> + if (json !is Map<*,*>) return@mapNotNull null + val roomID = json["room_id"] as? String ?: return@mapNotNull null + // A 401 means that we didn't provide a (valid) auth token for a route that required one. We use this as an + // indication that the token we're using has expired. Note that a 403 has a different meaning; it means that + // we provided a valid token but it doesn't have a high enough permission level for the route in question. + val statusCode = json["status_code"] as? Int ?: return@mapNotNull null if (statusCode == 401) { // delete auth token and return null - storage.removeAuthToken(roomId, server) + storage.removeAuthToken(roomID, server) } - - // check and store mods - val moderators = roomJson["moderators"] as? List ?: return@mapNotNull null - handleModerators("$server.$roomId", moderators) - - // get deletions + // Moderators + val moderators = json["moderators"] as? List ?: return@mapNotNull null + handleModerators("$server.$roomID", moderators) + // Deletions val type = TypeFactory.defaultInstance().constructCollectionType(List::class.java, MessageDeletion::class.java) - val idsAsString = JsonUtil.toJson(roomJson["deletions"]) - val deletedServerIDs = JsonUtil.fromJson>(idsAsString, type) ?: throw Error.PARSING_FAILED - val lastDeletionServerId = storage.getLastDeletionServerId(roomId, server) ?: 0 + val idsAsString = JsonUtil.toJson(json["deletions"]) + val deletedServerIDs = JsonUtil.fromJson>(idsAsString, type) ?: throw Error.ParsingFailed + val lastDeletionServerID = storage.getLastDeletionServerId(roomID, server) ?: 0 val serverID = deletedServerIDs.maxByOrNull {it.id } ?: MessageDeletion.EMPTY - if (serverID.id > lastDeletionServerId) { - storage.setLastDeletionServerId(roomId, server, serverID.id) + if (serverID.id > lastDeletionServerID) { + storage.setLastDeletionServerId(roomID, server, serverID.id) } - - // get messages - val rawMessages = roomJson["messages"] as? List> ?: return@mapNotNull null // parsing failed - - val lastMessageServerId = storage.getLastMessageServerId(roomId, server) ?: 0 - var currentMax = lastMessageServerId + // Messages + val rawMessages = json["messages"] as? List> ?: return@mapNotNull null + val lastMessageServerID = storage.getLastMessageServerId(roomID, server) ?: 0 + var currentLastMessageServerID = lastMessageServerID val messages = rawMessages.mapNotNull { rawMessage -> val message = OpenGroupMessageV2.fromJSON(rawMessage)?.apply { - currentMax = maxOf(currentMax,this.serverID ?: 0) + currentLastMessageServerID = maxOf(currentLastMessageServerID,this.serverID ?: 0) } + // TODO: We need to check the signature here... message } - storage.setLastMessageServerId(roomId, server, currentMax) - roomId to CompactPollResult( - messages = messages, - deletions = deletedServerIDs.map { it.deletedMessageId }, - moderators = moderators + storage.setLastMessageServerId(roomID, server, currentLastMessageServerID) + roomID to CompactPollResult( + messages = messages, + deletions = deletedServerIDs.map { it.deletedMessageId }, + moderators = moderators ) }.toMap() } @@ -443,7 +423,7 @@ object OpenGroupAPIV2 { val earlyGroups = groups.map { group -> DefaultGroup(group.id, group.name, null) } - // see if we have any cached rooms, and if they already have images, don't overwrite with early non-image results + // See if we have any cached rooms, and if they already have images don't overwrite them with early non-image results defaultRooms.replayCache.firstOrNull()?.let { replayed -> if (replayed.none { it.image?.isNotEmpty() == true}) { defaultRooms.tryEmit(earlyGroups) @@ -452,12 +432,11 @@ object OpenGroupAPIV2 { val images = groups.map { group -> group.id to downloadOpenGroupProfilePicture(group.id, DEFAULT_SERVER) }.toMap() - groups.map { group -> val image = try { images[group.id]!!.get() } catch (e: Exception) { - // no image or image failed to download + // No image or image failed to download null } DefaultGroup(group.id, group.name, image) @@ -470,9 +449,9 @@ object OpenGroupAPIV2 { fun getInfo(room: String, server: String): Promise { val request = Request(verb = GET, room = null, server = server, endpoint = "rooms/$room", isAuthRequired = false) return send(request).map { json -> - val rawRoom = json["room"] as? Map<*, *> ?: throw Error.PARSING_FAILED - val id = rawRoom["id"] as? String ?: throw Error.PARSING_FAILED - val name = rawRoom["name"] as? String ?: throw Error.PARSING_FAILED + val rawRoom = json["room"] as? Map<*, *> ?: throw Error.ParsingFailed + val id = rawRoom["id"] as? String ?: throw Error.ParsingFailed + val name = rawRoom["name"] as? String ?: throw Error.ParsingFailed val imageID = rawRoom["image_id"] as? String Info(id = id, name = name, imageID = imageID) } @@ -481,13 +460,13 @@ object OpenGroupAPIV2 { fun getAllRooms(server: String): Promise, Exception> { val request = Request(verb = GET, room = null, server = server, endpoint = "rooms", isAuthRequired = false) return send(request).map { json -> - val rawRooms = json["rooms"] as? List> ?: throw Error.PARSING_FAILED + val rawRooms = json["rooms"] as? List> ?: throw Error.ParsingFailed rawRooms.mapNotNull { val roomJson = it as? Map<*, *> ?: return@mapNotNull null val id = roomJson["id"] as? String ?: return@mapNotNull null val name = roomJson["name"] as? String ?: return@mapNotNull null - val imageId = roomJson["image_id"] as? String - Info(id, name, imageId) + val imageID = roomJson["image_id"] as? String + Info(id, name, imageID) } } } @@ -495,12 +474,11 @@ object OpenGroupAPIV2 { fun getMemberCount(room: String, server: String): Promise { val request = Request(verb = GET, room = room, server = server, endpoint = "member_count") return send(request).map { json -> - val memberCount = json["member_count"] as? Int ?: throw Error.PARSING_FAILED + val memberCount = json["member_count"] as? Int ?: throw Error.ParsingFailed val storage = MessagingModuleConfiguration.shared.storage storage.setUserCount(room, server, memberCount) memberCount } } // endregion - } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupMessageV2.kt b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupMessageV2.kt index 262c3d2a7b..1b75c1224b 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupMessageV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupMessageV2.kt @@ -9,14 +9,18 @@ import org.session.libsignal.utilities.logging.Log import org.whispersystems.curve25519.Curve25519 data class OpenGroupMessageV2( - val serverID: Long? = null, - val sender: String?, - val sentTimestamp: Long, - // The serialized protobuf in base64 encoding - val base64EncodedData: String, - // When sending a message, the sender signs the serialized protobuf with their private key so that - // a receiving user can verify that the message wasn't tampered with. - val base64EncodedSignature: String? = null + val serverID: Long? = null, + val sender: String?, + val sentTimestamp: Long, + /** + * The serialized protobuf in base64 encoding. + */ + val base64EncodedData: String, + /** + * When sending a message, the sender signs the serialized protobuf with their private key so that + * a receiving user can verify that the message wasn't tampered with. + */ + val base64EncodedSignature: String? = null ) { companion object { @@ -28,11 +32,12 @@ data class OpenGroupMessageV2( val serverID = json["server_id"] as? Int val sender = json["public_key"] as? String val base64EncodedSignature = json["signature"] as? String - return OpenGroupMessageV2(serverID = serverID?.toLong(), - sender = sender, - sentTimestamp = sentTimestamp, - base64EncodedData = base64EncodedData, - base64EncodedSignature = base64EncodedSignature + return OpenGroupMessageV2( + serverID = serverID?.toLong(), + sender = sender, + sentTimestamp = sentTimestamp, + base64EncodedData = base64EncodedData, + base64EncodedSignature = base64EncodedSignature ) } @@ -41,29 +46,26 @@ data class OpenGroupMessageV2( fun sign(): OpenGroupMessageV2? { if (base64EncodedData.isEmpty()) return null val (publicKey, privateKey) = MessagingModuleConfiguration.shared.storage.getUserKeyPair() ?: return null - - if (sender != publicKey) return null // only sign our own messages? - + if (sender != publicKey) return null val signature = try { curve.calculateSignature(privateKey, decode(base64EncodedData)) } catch (e: Exception) { - Log.e("Loki", "Couldn't sign OpenGroupV2Message", e) + Log.w("Loki", "Couldn't sign open group message.", e) return null } - return copy(base64EncodedSignature = Base64.encodeBytes(signature)) } fun toJSON(): Map { - val jsonMap = mutableMapOf("data" to base64EncodedData, "timestamp" to sentTimestamp) - serverID?.let { jsonMap["server_id"] = serverID } - sender?.let { jsonMap["public_key"] = sender } - base64EncodedSignature?.let { jsonMap["signature"] = base64EncodedSignature } - return jsonMap + val json = mutableMapOf( "data" to base64EncodedData, "timestamp" to sentTimestamp ) + serverID?.let { json["server_id"] = it } + sender?.let { json["public_key"] = it } + base64EncodedSignature?.let { json["signature"] = it } + return json } - fun toProto(): SignalServiceProtos.Content = decode(base64EncodedData).let(PushTransportDetails::getStrippedPaddingMessageBody).let { bytes -> - SignalServiceProtos.Content.parseFrom(bytes) + fun toProto(): SignalServiceProtos.Content { + val data = decode(base64EncodedData).let(PushTransportDetails::getStrippedPaddingMessageBody) + return SignalServiceProtos.Content.parseFrom(data) } - } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupV2.kt b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupV2.kt index 29965079cf..1e766a42ed 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupV2.kt @@ -1,51 +1,50 @@ package org.session.libsession.messaging.open_groups import org.session.libsignal.utilities.JsonUtil +import org.session.libsignal.utilities.logging.Log import java.util.* data class OpenGroupV2( - val server: String, - val room: String, - val id: String, - val name: String, - val publicKey: String + val server: String, + val room: String, + val id: String, + val name: String, + val publicKey: String ) { constructor(server: String, room: String, name: String, publicKey: String) : this( - server = server, - room = room, - id = "$server.$room", - name = name, - publicKey = publicKey, + server = server, + room = room, + id = "$server.$room", + name = name, + publicKey = publicKey, ) companion object { - fun fromJson(jsonAsString: String): OpenGroupV2? { + fun fromJSON(jsonAsString: String): OpenGroupV2? { return try { val json = JsonUtil.fromJson(jsonAsString) if (!json.has("room")) return null - - val room = json.get("room").asText().toLowerCase(Locale.getDefault()) - val server = json.get("server").asText().toLowerCase(Locale.getDefault()) + val room = json.get("room").asText().toLowerCase(Locale.US) + val server = json.get("server").asText().toLowerCase(Locale.US) val displayName = json.get("displayName").asText() val publicKey = json.get("publicKey").asText() - OpenGroupV2(server, room, displayName, publicKey) } catch (e: Exception) { + Log.w("Loki", "Couldn't parse open group from JSON: $jsonAsString.", e); null } } } - fun toJoinUrl(): String = "$server/$room?public_key=$publicKey" - fun toJson(): Map = mapOf( - "room" to room, - "server" to server, - "displayName" to name, - "publicKey" to publicKey, + "room" to room, + "server" to server, + "displayName" to name, + "publicKey" to publicKey, ) + val joinURL: String get() = "$server/$room?public_key=$publicKey" } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt index 2a0b13ae3e..1f8782311e 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt @@ -126,7 +126,7 @@ private fun handleConfigurationMessage(message: ConfigurationMessage) { handleNewClosedGroup(message.sender!!, message.sentTimestamp!!, closeGroup.publicKey, closeGroup.name, closeGroup.encryptionKeyPair!!, closeGroup.members, closeGroup.admins, message.sentTimestamp!!) } val allOpenGroups = storage.getAllOpenGroups().map { it.value.server } - val allV2OpenGroups = storage.getAllV2OpenGroups().map { it.value.toJoinUrl() } + val allV2OpenGroups = storage.getAllV2OpenGroups().map { it.value.joinURL } for (openGroup in message.openGroups) { if (allOpenGroups.contains(openGroup) || allV2OpenGroups.contains(openGroup)) continue storage.addOpenGroup(openGroup, 1) From c8cf5ebfa0f7187cc968810a8595ae4ecba03c78 Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 14:52:24 +1000 Subject: [PATCH 16/31] Make custom error messages actually work --- .../messaging/file_server/FileServerAPIV2.kt | 66 ++++++++----------- .../messaging/open_groups/OpenGroupAPIV2.kt | 24 ++----- 2 files changed, 34 insertions(+), 56 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt b/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt index 1762a797d9..c8db066692 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/file_server/FileServerAPIV2.kt @@ -15,77 +15,66 @@ import org.session.libsignal.utilities.logging.Log object FileServerAPIV2 { - const val DEFAULT_SERVER = "http://88.99.175.227" private const val DEFAULT_SERVER_PUBLIC_KEY = "7cb31905b55cd5580c686911debf672577b3fb0bff81df4ce2d5c4cb3a7aaa69" + const val DEFAULT_SERVER = "http://88.99.175.227" - sealed class Error : Exception() { - object PARSING_FAILED : Error() - object INVALID_URL : Error() - - fun errorDescription() = when (this) { - PARSING_FAILED -> "Invalid response." - INVALID_URL -> "Invalid URL." - } - + sealed class Error(message: String) : Exception(message) { + object ParsingFailed : Error("Invalid response.") + object InvalidURL : Error("Invalid URL.") } data class Request( - val verb: HTTP.Verb, - val endpoint: String, - val queryParameters: Map = mapOf(), - val parameters: Any? = null, - val headers: Map = mapOf(), - // Always `true` under normal circumstances. You might want to disable - // this when running over Lokinet. - val useOnionRouting: Boolean = true + val verb: HTTP.Verb, + val endpoint: String, + val queryParameters: Map = mapOf(), + val parameters: Any? = null, + val headers: Map = mapOf(), + /** + * Always `true` under normal circumstances. You might want to disable + * this when running over Lokinet. + */ + val useOnionRouting: Boolean = true ) private fun createBody(parameters: Any?): RequestBody? { if (parameters == null) return null - val parametersAsJSON = JsonUtil.toJson(parameters) return RequestBody.create(MediaType.get("application/json"), parametersAsJSON) } private fun send(request: Request): Promise, Exception> { - val parsed = HttpUrl.parse(DEFAULT_SERVER) ?: return Promise.ofFail(OpenGroupAPIV2.Error.InvalidURL) + val url = HttpUrl.parse(DEFAULT_SERVER) ?: return Promise.ofFail(OpenGroupAPIV2.Error.InvalidURL) val urlBuilder = HttpUrl.Builder() - .scheme(parsed.scheme()) - .host(parsed.host()) - .port(parsed.port()) - .addPathSegments(request.endpoint) - + .scheme(url.scheme()) + .host(url.host()) + .port(url.port()) + .addPathSegments(request.endpoint) if (request.verb == HTTP.Verb.GET) { for ((key, value) in request.queryParameters) { urlBuilder.addQueryParameter(key, value) } } - val requestBuilder = okhttp3.Request.Builder() - .url(urlBuilder.build()) - .headers(Headers.of(request.headers)) + .url(urlBuilder.build()) + .headers(Headers.of(request.headers)) when (request.verb) { HTTP.Verb.GET -> requestBuilder.get() HTTP.Verb.PUT -> requestBuilder.put(createBody(request.parameters)!!) HTTP.Verb.POST -> requestBuilder.post(createBody(request.parameters)!!) HTTP.Verb.DELETE -> requestBuilder.delete(createBody(request.parameters)) } - if (request.useOnionRouting) { - return OnionRequestAPI.sendOnionRequest(requestBuilder.build(), DEFAULT_SERVER, DEFAULT_SERVER_PUBLIC_KEY) - .fail { e -> - Log.e("Loki", "FileServerV2 failed with error",e) - } + return OnionRequestAPI.sendOnionRequest(requestBuilder.build(), DEFAULT_SERVER, DEFAULT_SERVER_PUBLIC_KEY).fail { e -> + Log.e("Loki", "File server request failed.", e) + } } else { return Promise.ofFail(IllegalStateException("It's currently not allowed to send non onion routed requests.")) } - } - // region Sending fun upload(file: ByteArray): Promise { val base64EncodedFile = Base64.encodeBytes(file) - val parameters = mapOf("file" to base64EncodedFile) + val parameters = mapOf( "file" to base64EncodedFile ) val request = Request(verb = HTTP.Verb.POST, endpoint = "files", parameters = parameters) return send(request).map { json -> json["result"] as? Long ?: throw OpenGroupAPIV2.Error.ParsingFailed @@ -95,9 +84,8 @@ object FileServerAPIV2 { fun download(file: Long): Promise { val request = Request(verb = HTTP.Verb.GET, endpoint = "files/$file") return send(request).map { json -> - val base64EncodedFile = json["result"] as? String ?: throw Error.PARSING_FAILED - Base64.decode(base64EncodedFile) ?: throw Error.PARSING_FAILED + val base64EncodedFile = json["result"] as? String ?: throw Error.ParsingFailed + Base64.decode(base64EncodedFile) ?: throw Error.ParsingFailed } } - } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt index 7f88cd3bc8..33f9a9613a 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt @@ -34,23 +34,13 @@ object OpenGroupAPIV2 { private const val DEFAULT_SERVER_PUBLIC_KEY = "a03c383cf63c3c4efe67acc52112a6dd734b3a946b9545f488aaa93da7991238" const val DEFAULT_SERVER = "http://116.203.70.33" - sealed class Error : Exception() { - object Generic : Error() - object ParsingFailed : Error() - object DecryptionFailed : Error() - object SigningFailed : Error() - object InvalidURL : Error() - object NoPublicKey : Error() - - fun errorDescription() = when (this) { - Error.Generic -> "An error occurred." - Error.ParsingFailed -> "Invalid response." - Error.DecryptionFailed -> "Couldn't decrypt response." - Error.SigningFailed -> "Couldn't sign message." - Error.InvalidURL -> "Invalid URL." - Error.NoPublicKey -> "Couldn't find server public key." - } - + sealed class Error(message: String) : Exception(message) { + object Generic : Error("An error occurred.") + object ParsingFailed : Error("Invalid response.") + object DecryptionFailed : Error("Couldn't decrypt response.") + object SigningFailed : Error("Couldn't sign message.") + object InvalidURL : Error("Invalid URL.") + object NoPublicKey : Error("Couldn't find server public key.") } data class DefaultGroup(val id: String, val name: String, val image: ByteArray?) { From 174bccb0b7781657b9588dcdce8af189bc2de95c Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 15:28:14 +1000 Subject: [PATCH 17/31] Fix missing signature validation --- .../messaging/MessagingModuleConfiguration.kt | 4 +- .../messaging/open_groups/OpenGroupAPIV2.kt | 99 +++++++++---------- 2 files changed, 50 insertions(+), 53 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/MessagingModuleConfiguration.kt b/libsession/src/main/java/org/session/libsession/messaging/MessagingModuleConfiguration.kt index e6ec434c9d..68610f9638 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/MessagingModuleConfiguration.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/MessagingModuleConfiguration.kt @@ -8,8 +8,8 @@ class MessagingModuleConfiguration( val context: Context, val storage: StorageProtocol, val messageDataProvider: MessageDataProvider, - val sessionProtocol: SessionProtocol) -{ + val sessionProtocol: SessionProtocol +) { companion object { lateinit var shared: MessagingModuleConfiguration diff --git a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt index 33f9a9613a..0884370c21 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/open_groups/OpenGroupAPIV2.kt @@ -65,19 +65,19 @@ object OpenGroupAPIV2 { } data class Request( - val verb: HTTP.Verb, - val room: String?, - val server: String, - val endpoint: String, - val queryParameters: Map = mapOf(), - val parameters: Any? = null, - val headers: Map = mapOf(), - val isAuthRequired: Boolean = true, - /** - * Always `true` under normal circumstances. You might want to disable - * this when running over Lokinet. - */ - val useOnionRouting: Boolean = true + val verb: HTTP.Verb, + val room: String?, + val server: String, + val endpoint: String, + val queryParameters: Map = mapOf(), + val parameters: Any? = null, + val headers: Map = mapOf(), + val isAuthRequired: Boolean = true, + /** + * Always `true` under normal circumstances. You might want to disable + * this when running over Lokinet. + */ + val useOnionRouting: Boolean = true ) private fun createBody(parameters: Any?): RequestBody? { @@ -241,36 +241,42 @@ object OpenGroupAPIV2 { queryParameters += "from_server_id" to lastId.toString() } val request = Request(verb = GET, room = room, server = server, endpoint = "messages", queryParameters = queryParameters) - return send(request).map { jsonList -> - @Suppress("UNCHECKED_CAST") val rawMessages = jsonList["messages"] as? List> + return send(request).map { json -> + @Suppress("UNCHECKED_CAST") val rawMessages = json["messages"] as? List> ?: throw Error.ParsingFailed - val lastMessageServerID = storage.getLastMessageServerId(room, server) ?: 0 - var currentLastMessageServerID = lastMessageServerID - val messages = rawMessages.mapNotNull { json -> - try { - val message = OpenGroupMessageV2.fromJSON(json) ?: return@mapNotNull null - if (message.serverID == null || message.sender.isNullOrEmpty()) return@mapNotNull null - val sender = message.sender - val data = decode(message.base64EncodedData) - val signature = decode(message.base64EncodedSignature) - val publicKey = Hex.fromStringCondensed(sender.removing05PrefixIfNeeded()) - val isValid = curve.verifySignature(publicKey, data, signature) - if (!isValid) { - Log.d("Loki", "Ignoring message with invalid signature") - return@mapNotNull null - } - if (message.serverID > lastMessageServerID) { - currentLastMessageServerID = message.serverID - } - message - } catch (e: Exception) { - null - } - } - storage.setLastMessageServerId(room, server, currentLastMessageServerID) - messages + parseMessages(room, server, rawMessages) } } + + private fun parseMessages(room: String, server: String, rawMessages: List>): List { + val storage = MessagingModuleConfiguration.shared.storage + val lastMessageServerID = storage.getLastMessageServerId(room, server) ?: 0 + var currentLastMessageServerID = lastMessageServerID + val messages = rawMessages.mapNotNull { json -> + json as Map + try { + val message = OpenGroupMessageV2.fromJSON(json) ?: return@mapNotNull null + if (message.serverID == null || message.sender.isNullOrEmpty()) return@mapNotNull null + val sender = message.sender + val data = decode(message.base64EncodedData) + val signature = decode(message.base64EncodedSignature) + val publicKey = Hex.fromStringCondensed(sender.removing05PrefixIfNeeded()) + val isValid = curve.verifySignature(publicKey, data, signature) + if (!isValid) { + Log.d("Loki", "Ignoring message with invalid signature.") + return@mapNotNull null + } + if (message.serverID > lastMessageServerID) { + currentLastMessageServerID = message.serverID + } + message + } catch (e: Exception) { + null + } + } + storage.setLastMessageServerId(room, server, currentLastMessageServerID) + return messages + } // endregion // region Message Deletion @@ -381,22 +387,13 @@ object OpenGroupAPIV2 { val idsAsString = JsonUtil.toJson(json["deletions"]) val deletedServerIDs = JsonUtil.fromJson>(idsAsString, type) ?: throw Error.ParsingFailed val lastDeletionServerID = storage.getLastDeletionServerId(roomID, server) ?: 0 - val serverID = deletedServerIDs.maxByOrNull {it.id } ?: MessageDeletion.EMPTY + val serverID = deletedServerIDs.maxByOrNull { it.id } ?: MessageDeletion.EMPTY if (serverID.id > lastDeletionServerID) { storage.setLastDeletionServerId(roomID, server, serverID.id) } // Messages val rawMessages = json["messages"] as? List> ?: return@mapNotNull null - val lastMessageServerID = storage.getLastMessageServerId(roomID, server) ?: 0 - var currentLastMessageServerID = lastMessageServerID - val messages = rawMessages.mapNotNull { rawMessage -> - val message = OpenGroupMessageV2.fromJSON(rawMessage)?.apply { - currentLastMessageServerID = maxOf(currentLastMessageServerID,this.serverID ?: 0) - } - // TODO: We need to check the signature here... - message - } - storage.setLastMessageServerId(roomID, server, currentLastMessageServerID) + val messages = parseMessages(roomID, server, rawMessages) roomID to CompactPollResult( messages = messages, deletions = deletedServerIDs.map { it.deletedMessageId }, From f5238982c3c199c987bd12faaa85bd12484b0293 Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 15:47:17 +1000 Subject: [PATCH 18/31] Add missing message padding --- .../libsession/messaging/sending_receiving/MessageSender.kt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/MessageSender.kt b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/MessageSender.kt index 84afeff63b..df7a0d7df8 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/MessageSender.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/MessageSender.kt @@ -258,11 +258,11 @@ object MessageSender { } val proto = message.toProto()!! - + val plaintext = PushTransportDetails.getPaddedMessageBody(proto.toByteArray()) val openGroupMessage = OpenGroupMessageV2( sender = message.sender, sentTimestamp = message.sentTimestamp!!, - base64EncodedData = Base64.encodeBytes(proto.toByteArray()), + base64EncodedData = Base64.encodeBytes(plaintext), ) OpenGroupAPIV2.send(openGroupMessage,room,server).success { From bb850cf99ef427b360e30e2d7e523f58dc6be389 Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 16:17:25 +1000 Subject: [PATCH 19/31] Minor job type refactoring --- .../loki/api/BackgroundPollWorker.kt | 1 + .../loki/database/SessionJobDatabase.kt | 38 ++++++------- .../messaging/jobs/AttachmentDownloadJob.kt | 24 +++++---- .../messaging/jobs/AttachmentUploadJob.kt | 43 +++++++-------- .../session/libsession/messaging/jobs/Job.kt | 9 ++-- .../libsession/messaging/jobs/JobQueue.kt | 27 +++++----- .../messaging/jobs/MessageReceiveJob.kt | 38 +++++++------ .../messaging/jobs/MessageSendJob.kt | 53 +++++++++++-------- .../messaging/jobs/NotifyPNServerJob.kt | 14 +++-- .../messaging/jobs/SessionJobInstantiator.kt | 6 +-- .../jobs/SessionJobManagerFactories.kt | 1 + 11 files changed, 136 insertions(+), 118 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt index 7b4f2c2aa6..a8d00d690c 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt @@ -75,6 +75,7 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor val userPublicKey = TextSecurePreferences.getLocalNumber(context)!! val privateChatsPromise = SnodeAPI.getMessages(userPublicKey).map { envelopes -> envelopes.map { envelope -> + // FIXME: Using a job here seems like a bad idea... MessageReceiveJob(envelope.toByteArray(), false).executeAsync() } } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt index 04edd3715e..3647a9937c 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt @@ -18,7 +18,8 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa const val jobType = "job_type" const val failureCount = "failure_count" const val serializedData = "serialized_data" - @JvmStatic val createSessionJobTableCommand = "CREATE TABLE $sessionJobTable ($jobID INTEGER PRIMARY KEY, $jobType STRING, $failureCount INTEGER DEFAULT 0, $serializedData TEXT);" + @JvmStatic val createSessionJobTableCommand + = "CREATE TABLE $sessionJobTable ($jobID INTEGER PRIMARY KEY, $jobType STRING, $failureCount INTEGER DEFAULT 0, $serializedData TEXT);" } fun persistJob(job: Job) { @@ -31,40 +32,41 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa database.insertOrUpdate(sessionJobTable, contentValues, "$jobID = ?", arrayOf(jobID)) } - fun markJobAsSucceeded(jobId: String) { - databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf(jobId)) + fun markJobAsSucceeded(jobID: String) { + databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf( jobID )) } - fun markJobAsFailed(jobId: String) { - databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf(jobId)) + fun markJobAsFailed(jobID: String) { + databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf( jobID )) } fun getAllPendingJobs(type: String): Map { val database = databaseHelper.readableDatabase - return database.getAll(sessionJobTable, "$jobType = ?", arrayOf(type)) { cursor -> - val jobId = cursor.getString(jobID) + return database.getAll(sessionJobTable, "$jobType = ?", arrayOf( type )) { cursor -> + val jobID = cursor.getString(jobID) try { - jobId to jobFromCursor(cursor) + jobID to jobFromCursor(cursor) } catch (e: Exception) { - Log.e("Loki", "Error serializing Job of type $type",e) - jobId to null + Log.e("Loki", "Error deserializing job of type: $type.", e) + jobID to null } }.toMap() } fun getAttachmentUploadJob(attachmentID: Long): AttachmentUploadJob? { val database = databaseHelper.readableDatabase - var result = mutableListOf() - database.getAll(sessionJobTable, "$jobType = ?", arrayOf(AttachmentUploadJob.KEY)) { cursor -> - result.add(jobFromCursor(cursor) as AttachmentUploadJob) + val result = mutableListOf() + database.getAll(sessionJobTable, "$jobType = ?", arrayOf( AttachmentUploadJob.KEY )) { cursor -> + val job = jobFromCursor(cursor) as AttachmentUploadJob? + if (job != null) { result.add(job) } } return result.firstOrNull { job -> job.attachmentID == attachmentID } } fun getMessageSendJob(messageSendJobID: String): MessageSendJob? { val database = databaseHelper.readableDatabase - return database.get(sessionJobTable, "$jobID = ? AND $jobType = ?", arrayOf(messageSendJobID, MessageSendJob.KEY)) { cursor -> - jobFromCursor(cursor) as MessageSendJob + return database.get(sessionJobTable, "$jobID = ? AND $jobType = ?", arrayOf( messageSendJobID, MessageSendJob.KEY )) { cursor -> + jobFromCursor(cursor) as MessageSendJob? } } @@ -72,7 +74,7 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa val database = databaseHelper.readableDatabase var cursor: android.database.Cursor? = null try { - cursor = database.rawQuery("SELECT * FROM $sessionJobTable WHERE $jobID = ?", arrayOf(job.id)) + cursor = database.rawQuery("SELECT * FROM $sessionJobTable WHERE $jobID = ?", arrayOf( job.id )) return cursor != null && cursor.moveToFirst() } catch (e: Exception) { // Do nothing @@ -82,10 +84,10 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa return false } - private fun jobFromCursor(cursor: Cursor): Job { + private fun jobFromCursor(cursor: Cursor): Job? { val type = cursor.getString(jobType) val data = SessionJobHelper.dataSerializer.deserialize(cursor.getString(serializedData)) - val job = SessionJobHelper.sessionJobInstantiator.instantiate(type, data) + val job = SessionJobHelper.sessionJobInstantiator.instantiate(type, data) ?: return null job.id = cursor.getString(jobID) job.failureCount = cursor.getInt(failureCount) return job diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt index 6855b08b02..8285813206 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt @@ -3,7 +3,6 @@ package org.session.libsession.messaging.jobs import okhttp3.HttpUrl import org.session.libsession.messaging.MessagingModuleConfiguration import org.session.libsession.messaging.file_server.FileServerAPI -import org.session.libsession.messaging.file_server.FileServerAPIV2 import org.session.libsession.messaging.open_groups.OpenGroupAPIV2 import org.session.libsession.messaging.sending_receiving.attachments.AttachmentState import org.session.libsession.messaging.utilities.DotNetAPI @@ -31,8 +30,8 @@ class AttachmentDownloadJob(val attachmentID: Long, val databaseMessageID: Long) val KEY: String = "AttachmentDownloadJob" // Keys used for database storage - private val KEY_ATTACHMENT_ID = "attachment_id" - private val KEY_TS_INCOMING_MESSAGE_ID = "tsIncoming_message_id" + private val ATTACHMENT_ID_KEY = "attachment_id" + private val TS_INCOMING_MESSAGE_ID_KEY = "tsIncoming_message_id" } override fun execute() { @@ -52,18 +51,19 @@ class AttachmentDownloadJob(val attachmentID: Long, val databaseMessageID: Long) try { val messageDataProvider = MessagingModuleConfiguration.shared.messageDataProvider val attachment = messageDataProvider.getDatabaseAttachment(attachmentID) - ?: return handleFailure(Error.NoAttachment) + ?: return handleFailure(Error.NoAttachment) messageDataProvider.setAttachmentState(AttachmentState.STARTED, attachmentID, this.databaseMessageID) val tempFile = createTempFile() - val threadId = MessagingModuleConfiguration.shared.storage.getThreadIdForMms(databaseMessageID) val openGroupV2 = MessagingModuleConfiguration.shared.storage.getV2OpenGroup(threadId.toString()) - val stream = if (openGroupV2 == null) { DownloadUtilities.downloadFile(tempFile, attachment.url, FileServerAPI.maxFileSize, null) // Assume we're retrieving an attachment for an open group server if the digest is not set - if (attachment.digest?.size ?: 0 == 0 || attachment.key.isNullOrEmpty()) FileInputStream(tempFile) - else AttachmentCipherInputStream.createForAttachment(tempFile, attachment.size, Base64.decode(attachment.key), attachment.digest) + if (attachment.digest?.size ?: 0 == 0 || attachment.key.isNullOrEmpty()) { + FileInputStream(tempFile) + } else { + AttachmentCipherInputStream.createForAttachment(tempFile, attachment.size, Base64.decode(attachment.key), attachment.digest) + } } else { val url = HttpUrl.parse(attachment.url)!! val fileId = url.pathSegments().last() @@ -100,8 +100,9 @@ class AttachmentDownloadJob(val attachmentID: Long, val databaseMessageID: Long) } override fun serialize(): Data { - return Data.Builder().putLong(KEY_ATTACHMENT_ID, attachmentID) - .putLong(KEY_TS_INCOMING_MESSAGE_ID, databaseMessageID) + return Data.Builder() + .putLong(ATTACHMENT_ID_KEY, attachmentID) + .putLong(TS_INCOMING_MESSAGE_ID_KEY, databaseMessageID) .build(); } @@ -110,8 +111,9 @@ class AttachmentDownloadJob(val attachmentID: Long, val databaseMessageID: Long) } class Factory : Job.Factory { + override fun create(data: Data): AttachmentDownloadJob { - return AttachmentDownloadJob(data.getLong(KEY_ATTACHMENT_ID), data.getLong(KEY_TS_INCOMING_MESSAGE_ID)) + return AttachmentDownloadJob(data.getLong(ATTACHMENT_ID_KEY), data.getLong(TS_INCOMING_MESSAGE_ID_KEY)) } } } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt index 1d8b1a7170..e0849d1daf 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt @@ -30,44 +30,39 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess // Settings override val maxFailureCount: Int = 20 + companion object { val TAG = AttachmentUploadJob::class.simpleName val KEY: String = "AttachmentUploadJob" // Keys used for database storage - private val KEY_ATTACHMENT_ID = "attachment_id" - private val KEY_THREAD_ID = "thread_id" - private val KEY_MESSAGE = "message" - private val KEY_MESSAGE_SEND_JOB_ID = "message_send_job_id" + private val ATTACHMENT_ID_KEY = "attachment_id" + private val THREAD_ID_KEY = "thread_id" + private val MESSAGE_KEY = "message" + private val MESSAGE_SEND_JOB_ID_KEY = "message_send_job_id" } override fun execute() { try { val attachment = MessagingModuleConfiguration.shared.messageDataProvider.getScaledSignalAttachmentStream(attachmentID) ?: return handleFailure(Error.NoAttachment) - val usePadding = false val openGroupV2 = MessagingModuleConfiguration.shared.storage.getV2OpenGroup(threadID) val openGroup = MessagingModuleConfiguration.shared.storage.getOpenGroup(threadID) - val server = openGroup?.let { - it.server - } ?: openGroupV2?.let { - it.server - } ?: FileServerAPI.shared.server + val server = openGroupV2?.server ?: openGroup?.server ?: FileServerAPI.shared.server val shouldEncrypt = (openGroup == null && openGroupV2 == null) // Encrypt if this isn't an open group - val attachmentKey = Util.getSecretBytes(64) val paddedLength = if (usePadding) PaddingInputStream.getPaddedSize(attachment.length) else attachment.length val dataStream = if (usePadding) PaddingInputStream(attachment.inputStream, attachment.length) else attachment.inputStream val ciphertextLength = if (shouldEncrypt) AttachmentCipherOutputStream.getCiphertextLength(paddedLength) else attachment.length - val outputStreamFactory = if (shouldEncrypt) AttachmentCipherOutputStreamFactory(attachmentKey) else PlaintextOutputStreamFactory() val attachmentData = PushAttachmentData(attachment.contentType, dataStream, ciphertextLength, outputStreamFactory, attachment.listener) - - val uploadResult = if (openGroupV2 == null) FileServerAPI.shared.uploadAttachment(server, attachmentData) else { + val uploadResult = if (openGroupV2 != null) { val dataBytes = attachmentData.data.readBytes() val result = OpenGroupAPIV2.upload(dataBytes, openGroupV2.room, openGroupV2.server).get() DotNetAPI.UploadResult(result, "${openGroupV2.server}/files/$result", byteArrayOf()) + } else { + FileServerAPI.shared.uploadAttachment(server, attachmentData) } handleSuccess(attachment, attachmentKey, uploadResult) } catch (e: java.lang.Exception) { @@ -82,7 +77,7 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess } private fun handleSuccess(attachment: SignalServiceAttachmentStream, attachmentKey: ByteArray, uploadResult: DotNetAPI.UploadResult) { - Log.w(TAG, "Attachment uploaded successfully.") + Log.d(TAG, "Attachment uploaded successfully.") delegate?.handleJobSucceeded(this) MessagingModuleConfiguration.shared.messageDataProvider.updateAttachmentAfterUploadSucceeded(attachmentID, attachment, attachmentKey, uploadResult) MessagingModuleConfiguration.shared.storage.resumeMessageSendJobIfNeeded(messageSendJobID) @@ -119,10 +114,11 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess val output = Output(serializedMessage) kryo.writeObject(output, message) output.close() - return Data.Builder().putLong(KEY_ATTACHMENT_ID, attachmentID) - .putString(KEY_THREAD_ID, threadID) - .putByteArray(KEY_MESSAGE, serializedMessage) - .putString(KEY_MESSAGE_SEND_JOB_ID, messageSendJobID) + return Data.Builder() + .putLong(ATTACHMENT_ID_KEY, attachmentID) + .putString(THREAD_ID_KEY, threadID) + .putByteArray(MESSAGE_KEY, serializedMessage) + .putString(MESSAGE_SEND_JOB_ID_KEY, messageSendJobID) .build(); } @@ -133,12 +129,17 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess class Factory: Job.Factory { override fun create(data: Data): AttachmentUploadJob { - val serializedMessage = data.getByteArray(KEY_MESSAGE) + val serializedMessage = data.getByteArray(MESSAGE_KEY) val kryo = Kryo() val input = Input(serializedMessage) val message: Message = kryo.readObject(input, Message::class.java) input.close() - return AttachmentUploadJob(data.getLong(KEY_ATTACHMENT_ID), data.getString(KEY_THREAD_ID)!!, message, data.getString(KEY_MESSAGE_SEND_JOB_ID)!!) + return AttachmentUploadJob( + data.getLong(ATTACHMENT_ID_KEY), + data.getString(THREAD_ID_KEY)!!, + message, + data.getString(MESSAGE_SEND_JOB_ID_KEY)!! + ) } } } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt index 4693fddf4a..b680734bb8 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt @@ -8,21 +8,20 @@ interface Job { val maxFailureCount: Int companion object { + // Keys used for database storage - private val KEY_ID = "id" - private val KEY_FAILURE_COUNT = "failure_count" + private val ID_KEY = "id" + private val FAILURE_COUNT_KEY = "failure_count" } fun execute() fun serialize(): Data - /** - * Returns the key that can be used to find the relevant factory needed to create your job. - */ fun getFactoryKey(): String interface Factory { + fun create(data: Data): T } } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt index cffb2db7d6..a89338cf3c 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt @@ -44,14 +44,15 @@ class JobQueue : JobDelegate { } companion object { + @JvmStatic val shared: JobQueue by lazy { JobQueue() } } private fun Job.canExecuteParallel(): Boolean { return this.javaClass in arrayOf( - AttachmentUploadJob::class.java, - AttachmentDownloadJob::class.java + AttachmentUploadJob::class.java, + AttachmentDownloadJob::class.java ) } @@ -68,7 +69,6 @@ class JobQueue : JobDelegate { val currentTime = System.currentTimeMillis() jobTimestampMap.putIfAbsent(currentTime, AtomicInteger()) job.id = currentTime.toString() + jobTimestampMap[currentTime]!!.getAndIncrement().toString() - MessagingModuleConfiguration.shared.storage.persistJob(job) } @@ -78,25 +78,26 @@ class JobQueue : JobDelegate { return } hasResumedPendingJobs = true - val allJobTypes = listOf(AttachmentUploadJob.KEY, - AttachmentDownloadJob.KEY, - MessageReceiveJob.KEY, - MessageSendJob.KEY, - NotifyPNServerJob.KEY + val allJobTypes = listOf( + AttachmentUploadJob.KEY, + AttachmentDownloadJob.KEY, + MessageReceiveJob.KEY, + MessageSendJob.KEY, + NotifyPNServerJob.KEY ) allJobTypes.forEach { type -> val allPendingJobs = MessagingModuleConfiguration.shared.storage.getAllPendingJobs(type) val pendingJobs = mutableListOf() for ((id, job) in allPendingJobs) { if (job == null) { - // job failed to serialize, remove it from the DB + // Job failed to deserialize, remove it from the DB handleJobFailedPermanently(id) } else { pendingJobs.add(job) } } pendingJobs.sortedBy { it.id }.forEach { job -> - Log.i("Jobs", "Resuming pending job of type: ${job::class.simpleName}.") + Log.i("Loki", "Resuming pending job of type: ${job::class.simpleName}.") queue.offer(job) // Offer always called on unlimited capacity } } @@ -110,15 +111,15 @@ class JobQueue : JobDelegate { override fun handleJobFailed(job: Job, error: Exception) { job.failureCount += 1 val storage = MessagingModuleConfiguration.shared.storage - if (storage.isJobCanceled(job)) { return Log.i("Jobs", "${job::class.simpleName} canceled.")} + if (storage.isJobCanceled(job)) { return Log.i("Loki", "${job::class.simpleName} canceled.")} if (job.failureCount == job.maxFailureCount) { handleJobFailedPermanently(job, error) } else { storage.persistJob(job) val retryInterval = getRetryInterval(job) - Log.i("Jobs", "${job::class.simpleName} failed; scheduling retry (failure count is ${job.failureCount}).") + Log.i("Loki", "${job::class.simpleName} failed; scheduling retry (failure count is ${job.failureCount}).") timer.schedule(delay = retryInterval) { - Log.i("Jobs", "Retrying ${job::class.simpleName}.") + Log.i("Loki", "Retrying ${job::class.simpleName}.") queue.offer(job) } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt index 7c527bebbf..ebca416250 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt @@ -11,7 +11,6 @@ class MessageReceiveJob(val data: ByteArray, val isBackgroundPoll: Boolean, val override var id: String? = null override var failureCount: Int = 0 - // Settings override val maxFailureCount: Int = 10 companion object { val TAG = MessageReceiveJob::class.simpleName @@ -20,10 +19,11 @@ class MessageReceiveJob(val data: ByteArray, val isBackgroundPoll: Boolean, val private val RECEIVE_LOCK = Object() // Keys used for database storage - private val KEY_DATA = "data" - private val KEY_IS_BACKGROUND_POLL = "is_background_poll" - private val KEY_OPEN_GROUP_MESSAGE_SERVER_ID = "openGroupMessageServerID" - private val KEY_OPEN_GROUP_ID = "open_group_id" + private val DATA_KEY = "data" + // FIXME: We probably shouldn't be using this job when background polling + private val IS_BACKGROUND_POLL_KEY = "is_background_poll" + private val OPEN_GROUP_MESSAGE_SERVER_ID_KEY = "openGroupMessageServerID" + private val OPEN_GROUP_ID_KEY = "open_group_id" } override fun execute() { @@ -35,19 +35,18 @@ class MessageReceiveJob(val data: ByteArray, val isBackgroundPoll: Boolean, val try { val isRetry: Boolean = failureCount != 0 val (message, proto) = MessageReceiver.parse(this.data, this.openGroupMessageServerID, isRetry) - synchronized(RECEIVE_LOCK) { + synchronized(RECEIVE_LOCK) { // FIXME: Do we need this? MessageReceiver.handle(message, proto, this.openGroupID) } this.handleSuccess() deferred.resolve(Unit) } catch (e: Exception) { - Log.e(TAG, "Couldn't receive message due to error", e) - val error = e as? MessageReceiver.Error - if (error != null && !error.isRetryable) { - Log.e("Loki", "Message receive job permanently failed due to error", e) - this.handlePermanentFailure(error) + Log.e(TAG, "Couldn't receive message.", e) + if (e is MessageReceiver.Error && !e.isRetryable) { + Log.e("Loki", "Message receive job permanently failed.", e) + this.handlePermanentFailure(e) } else { - Log.e("Loki", "Couldn't receive message due to error", e) + Log.e("Loki", "Couldn't receive message.", e) this.handleFailure(e) } deferred.resolve(Unit) // The promise is just used to keep track of when we're done @@ -68,10 +67,10 @@ class MessageReceiveJob(val data: ByteArray, val isBackgroundPoll: Boolean, val } override fun serialize(): Data { - val builder = Data.Builder().putByteArray(KEY_DATA, data) - .putBoolean(KEY_IS_BACKGROUND_POLL, isBackgroundPoll) - openGroupMessageServerID?.let { builder.putLong(KEY_OPEN_GROUP_MESSAGE_SERVER_ID, openGroupMessageServerID) } - openGroupID?.let { builder.putString(KEY_OPEN_GROUP_ID, openGroupID) } + val builder = Data.Builder().putByteArray(DATA_KEY, data) + .putBoolean(IS_BACKGROUND_POLL_KEY, isBackgroundPoll) + openGroupMessageServerID?.let { builder.putLong(OPEN_GROUP_MESSAGE_SERVER_ID_KEY, it) } + openGroupID?.let { builder.putString(OPEN_GROUP_ID_KEY, it) } return builder.build(); } @@ -82,7 +81,12 @@ class MessageReceiveJob(val data: ByteArray, val isBackgroundPoll: Boolean, val class Factory: Job.Factory { override fun create(data: Data): MessageReceiveJob { - return MessageReceiveJob(data.getByteArray(KEY_DATA), data.getBoolean(KEY_IS_BACKGROUND_POLL), data.getLong(KEY_OPEN_GROUP_MESSAGE_SERVER_ID), data.getString(KEY_OPEN_GROUP_ID)) + return MessageReceiveJob( + data.getByteArray(DATA_KEY), + data.getBoolean(IS_BACKGROUND_POLL_KEY), + data.getLong(OPEN_GROUP_MESSAGE_SERVER_ID_KEY), + data.getString(OPEN_GROUP_ID_KEY) + ) } } } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index 83822c4fc7..48266764a7 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -15,22 +15,22 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { override var id: String? = null override var failureCount: Int = 0 - // Settings override val maxFailureCount: Int = 10 + companion object { val TAG = MessageSendJob::class.simpleName val KEY: String = "MessageSendJob" // Keys used for database storage - private val KEY_MESSAGE = "message" - private val KEY_DESTINATION = "destination" + private val MESSAGE_KEY = "message" + private val DESTINATION_KEY = "destination" } override fun execute() { val messageDataProvider = MessagingModuleConfiguration.shared.messageDataProvider val message = message as? VisibleMessage - message?.let { - if(!messageDataProvider.isOutgoingMessage(message.sentTimestamp!!)) return // The message has been deleted + if (message != null) { + if (!messageDataProvider.isOutgoingMessage(message.sentTimestamp!!)) return // The message has been deleted val attachmentIDs = mutableListOf() attachmentIDs.addAll(message.attachmentIDs) message.quote?.let { it.attachmentID?.let { attachmentID -> attachmentIDs.add(attachmentID) } } @@ -51,9 +51,8 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { this.handleSuccess() }.fail { exception -> Log.e(TAG, "Couldn't send message due to error: $exception.") - val e = exception as? MessageSender.Error - e?.let { - if (!e.isRetryable) this.handlePermanentFailure(e) + if (exception is MessageSender.Error) { + if (!exception.isRetryable) { this.handlePermanentFailure(exception) } } this.handleFailure(exception) } @@ -70,8 +69,10 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { private fun handleFailure(error: Exception) { Log.w(TAG, "Failed to send $message::class.simpleName.") val message = message as? VisibleMessage - message?.let { - if(!MessagingModuleConfiguration.shared.messageDataProvider.isOutgoingMessage(message.sentTimestamp!!)) return // The message has been deleted + if (message != null) { + if (!MessagingModuleConfiguration.shared.messageDataProvider.isOutgoingMessage(message.sentTimestamp!!)) { + return // The message has been deleted + } } delegate?.handleJobFailed(this, error) } @@ -80,34 +81,42 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { val kryo = Kryo() kryo.isRegistrationRequired = false val output = Output(ByteArray(4096), 10_000_000) + // Message kryo.writeClassAndObject(output, message) output.close() val serializedMessage = output.toBytes() output.clear() + // Destination kryo.writeClassAndObject(output, destination) output.close() val serializedDestination = output.toBytes() - return Data.Builder().putByteArray(KEY_MESSAGE, serializedMessage) - .putByteArray(KEY_DESTINATION, serializedDestination) - .build(); + output.clear() + // Serialize + return Data.Builder() + .putByteArray(MESSAGE_KEY, serializedMessage) + .putByteArray(DESTINATION_KEY, serializedDestination) + .build() } override fun getFactoryKey(): String { return KEY } - class Factory: Job.Factory { + class Factory : Job.Factory { override fun create(data: Data): MessageSendJob { - val serializedMessage = data.getByteArray(KEY_MESSAGE) - val serializedDestination = data.getByteArray(KEY_DESTINATION) + val serializedMessage = data.getByteArray(MESSAGE_KEY) + val serializedDestination = data.getByteArray(DESTINATION_KEY) val kryo = Kryo() - var input = Input(serializedMessage) - val message = kryo.readClassAndObject(input) as Message - input.close() - input = Input(serializedDestination) - val destination = kryo.readClassAndObject(input) as Destination - input.close() + // Message + val messageInput = Input(serializedMessage) + val message = kryo.readClassAndObject(messageInput) as Message + messageInput.close() + // Destination + val destinationInput = Input(serializedDestination) + val destination = kryo.readClassAndObject(destinationInput) as Destination + destinationInput.close() + // Return return MessageSendJob(message, destination) } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt index fb99f54f56..d5a1674bad 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt @@ -21,16 +21,14 @@ class NotifyPNServerJob(val message: SnodeMessage) : Job { override var id: String? = null override var failureCount: Int = 0 - // Settings override val maxFailureCount: Int = 20 companion object { val KEY: String = "NotifyPNServerJob" // Keys used for database storage - private val KEY_MESSAGE = "message" + private val MESSAGE_KEY = "message" } - // Running override fun execute() { val server = PushNotificationAPI.server val parameters = mapOf( "data" to message.data, "send_to" to message.recipient ) @@ -41,10 +39,10 @@ class NotifyPNServerJob(val message: SnodeMessage) : Job { OnionRequestAPI.sendOnionRequest(request.build(), server, PushNotificationAPI.serverPublicKey, "/loki/v2/lsrpc").map { json -> val code = json["code"] as? Int if (code == null || code == 0) { - Log.d("Loki", "[Loki] Couldn't notify PN server due to error: ${json["message"] as? String ?: "null"}.") + Log.d("Loki", "Couldn't notify PN server due to error: ${json["message"] as? String ?: "null"}.") } }.fail { exception -> - Log.d("Loki", "[Loki] Couldn't notify PN server due to error: $exception.") + Log.d("Loki", "Couldn't notify PN server due to error: $exception.") } }.success { handleSuccess() @@ -68,17 +66,17 @@ class NotifyPNServerJob(val message: SnodeMessage) : Job { val output = Output(serializedMessage) kryo.writeObject(output, message) output.close() - return Data.Builder().putByteArray(KEY_MESSAGE, serializedMessage).build(); + return Data.Builder().putByteArray(MESSAGE_KEY, serializedMessage).build(); } override fun getFactoryKey(): String { return KEY } - class Factory: Job.Factory { + class Factory : Job.Factory { override fun create(data: Data): NotifyPNServerJob { - val serializedMessage = data.getByteArray(KEY_MESSAGE) + val serializedMessage = data.getByteArray(MESSAGE_KEY) val kryo = Kryo() val input = Input(serializedMessage) val message: SnodeMessage = kryo.readObject(input, SnodeMessage::class.java) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt index bf0a1b2f8a..a6336e9148 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt @@ -2,11 +2,11 @@ package org.session.libsession.messaging.jobs class SessionJobInstantiator(private val jobFactories: Map>) { - fun instantiate(jobFactoryKey: String, data: Data): Job { + fun instantiate(jobFactoryKey: String, data: Data): Job? { if (jobFactories.containsKey(jobFactoryKey)) { - return jobFactories[jobFactoryKey]?.create(data) ?: throw IllegalStateException("Tried to instantiate a job with key '$jobFactoryKey', but no matching factory was found.") + return jobFactories[jobFactoryKey]?.create(data) } else { - throw IllegalStateException("Tried to instantiate a job with key '$jobFactoryKey', but no matching factory was found.") + return null } } } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobManagerFactories.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobManagerFactories.kt index e7c02361e1..c681a67f3d 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobManagerFactories.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobManagerFactories.kt @@ -3,6 +3,7 @@ package org.session.libsession.messaging.jobs class SessionJobManagerFactories { companion object { + fun getSessionJobFactories(): Map> { return mapOf( AttachmentDownloadJob.KEY to AttachmentDownloadJob.Factory(), From c5e0589751a97ae1af938631023e0c87cc15c767 Mon Sep 17 00:00:00 2001 From: nielsandriesse Date: Wed, 12 May 2021 16:21:53 +1000 Subject: [PATCH 20/31] Don't crash on unexpected deserialization error --- .../session/libsession/messaging/jobs/Job.kt | 4 ++-- .../messaging/jobs/MessageSendJob.kt | 18 +++++++++++++++--- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt index b680734bb8..332eca002d 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt @@ -21,7 +21,7 @@ interface Job { fun getFactoryKey(): String interface Factory { - - fun create(data: Data): T + + fun create(data: Data): T? } } \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index 48266764a7..26620b3057 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -104,17 +104,29 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { class Factory : Job.Factory { - override fun create(data: Data): MessageSendJob { + override fun create(data: Data): MessageSendJob? { val serializedMessage = data.getByteArray(MESSAGE_KEY) val serializedDestination = data.getByteArray(DESTINATION_KEY) val kryo = Kryo() // Message val messageInput = Input(serializedMessage) - val message = kryo.readClassAndObject(messageInput) as Message + val message: Message + try { + message = kryo.readClassAndObject(messageInput) as Message + } catch (e: Exception) { + Log.e("Loki", "Couldn't deserialize message send job.", e) + return null + } messageInput.close() // Destination val destinationInput = Input(serializedDestination) - val destination = kryo.readClassAndObject(destinationInput) as Destination + val destination: Destination + try { + destination = kryo.readClassAndObject(destinationInput) as Destination + } catch (e: Exception) { + Log.e("Loki", "Couldn't deserialize message send job.", e) + return null + } destinationInput.close() // Return return MessageSendJob(message, destination) From edc1454609180dd8690dca22a8cff5da34dfb8dc Mon Sep 17 00:00:00 2001 From: jubb Date: Wed, 12 May 2021 16:48:18 +1000 Subject: [PATCH 21/31] fix: unnamed open groups being processed by creating new threads after deletion job db not marking successful/unsuccessful properly handling send and receive better / in order --- .../securesms/database/Storage.kt | 2 +- .../loki/database/SessionJobDatabase.kt | 8 +-- .../libsession/messaging/jobs/JobQueue.kt | 59 ++++++++++++------- .../messaging/jobs/MessageSendJob.kt | 8 ++- .../ReceivedMessageHandler.kt | 7 ++- 5 files changed, 55 insertions(+), 29 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt index 6a2ff6e8df..f93670ae88 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt @@ -581,7 +581,7 @@ class Storage(context: Context, helper: SQLCipherOpenHelper) : Database(context, val database = DatabaseFactory.getThreadDatabase(context) if (!openGroupID.isNullOrEmpty()) { val recipient = Recipient.from(context, Address.fromSerialized(GroupUtil.getEncodedOpenGroupID(openGroupID.toByteArray())), false) - return database.getOrCreateThreadIdFor(recipient) + return database.getThreadIdIfExistsFor(recipient) } else if (!groupPublicKey.isNullOrEmpty()) { val recipient = Recipient.from(context, Address.fromSerialized(GroupUtil.doubleEncodeGroupID(groupPublicKey)), false) return database.getOrCreateThreadIdFor(recipient) diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt index 3647a9937c..d8c072dd5f 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt @@ -29,15 +29,15 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa contentValues.put(jobType, job.getFactoryKey()) contentValues.put(failureCount, job.failureCount) contentValues.put(serializedData, SessionJobHelper.dataSerializer.serialize(job.serialize())) - database.insertOrUpdate(sessionJobTable, contentValues, "$jobID = ?", arrayOf(jobID)) + database.insertOrUpdate(sessionJobTable, contentValues, "$jobID = ?", arrayOf(job.id!!)) } fun markJobAsSucceeded(jobID: String) { - databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf( jobID )) + databaseHelper.writableDatabase.delete(sessionJobTable, "${Companion.jobID} = ?", arrayOf( jobID )) } fun markJobAsFailed(jobID: String) { - databaseHelper.writableDatabase.delete(sessionJobTable, "$jobID = ?", arrayOf( jobID )) + databaseHelper.writableDatabase.delete(sessionJobTable, "${Companion.jobID} = ?", arrayOf( jobID )) } fun getAllPendingJobs(type: String): Map { @@ -75,7 +75,7 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa var cursor: android.database.Cursor? = null try { cursor = database.rawQuery("SELECT * FROM $sessionJobTable WHERE $jobID = ?", arrayOf( job.id )) - return cursor != null && cursor.moveToFirst() + return cursor == null || !cursor.moveToFirst() } catch (e: Exception) { // Do nothing } finally { diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt index 246a766fce..fab49384fd 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt @@ -17,29 +17,50 @@ import kotlin.math.roundToLong class JobQueue : JobDelegate { private var hasResumedPendingJobs = false // Just for debugging private val jobTimestampMap = ConcurrentHashMap() - private val dispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher() - private val multiDispatcher = Executors.newFixedThreadPool(2).asCoroutineDispatcher() + private val rxDispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher() + private val txDispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher() + private val attachmentDispatcher = Executors.newFixedThreadPool(2).asCoroutineDispatcher() private val scope = GlobalScope + SupervisorJob() private val queue = Channel(UNLIMITED) val timer = Timer() + private fun CoroutineScope.processWithDispatcher(channel: Channel, dispatcher: CoroutineDispatcher) = launch(dispatcher) { + for (job in channel) { + if (!isActive) break + job.delegate = this@JobQueue + job.execute() + } + } + init { // Process jobs - scope.launch(dispatcher) { + scope.launch { + val rxQueue = Channel(capacity = 1024) + val txQueue = Channel(capacity = 1024) + val attachmentQueue = Channel(capacity = 1024) + + val receiveJob = processWithDispatcher(rxQueue, rxDispatcher) + val txJob = processWithDispatcher(txQueue, txDispatcher) + val attachmentJob = processWithDispatcher(attachmentQueue, attachmentDispatcher) + while (isActive) { - queue.receive().let { job -> - if (job.canExecuteParallel()) { - launch(multiDispatcher) { - job.delegate = this@JobQueue - job.execute() - } - } else { - job.delegate = this@JobQueue - job.execute() + for (job in queue) { + when (job) { + is NotifyPNServerJob, + is AttachmentUploadJob, + is MessageSendJob -> txQueue.send(job) + is AttachmentDownloadJob -> attachmentQueue.send(job) + else -> rxQueue.send(job) } } } + + // job has been cancelled + receiveJob.cancel() + txJob.cancel() + attachmentJob.cancel() + } } @@ -49,14 +70,6 @@ class JobQueue : JobDelegate { val shared: JobQueue by lazy { JobQueue() } } - private fun Job.canExecuteParallel(): Boolean { - return this.javaClass in arrayOf( - MessageSendJob::class.java, - AttachmentUploadJob::class.java, - AttachmentDownloadJob::class.java - ) - } - fun add(job: Job) { addWithoutExecuting(job) queue.offer(job) // offer always called on unlimited capacity @@ -112,8 +125,10 @@ class JobQueue : JobDelegate { override fun handleJobFailed(job: Job, error: Exception) { job.failureCount += 1 val storage = MessagingModuleConfiguration.shared.storage - if (storage.isJobCanceled(job)) { return Log.i("Loki", "${job::class.simpleName} canceled.")} - if (job.failureCount == job.maxFailureCount) { + if (storage.isJobCanceled(job)) { + return Log.i("Loki", "${job::class.simpleName} canceled.") + } + if (job.failureCount >= job.maxFailureCount) { handleJobFailedPermanently(job, error) } else { storage.persistJob(job) diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index 6c187b686a..1a0e4e57f4 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -12,6 +12,9 @@ import org.session.libsession.messaging.sending_receiving.MessageSender import org.session.libsignal.utilities.logging.Log class MessageSendJob(val message: Message, val destination: Destination) : Job { + + object AwaitingUploadException: Exception("Awaiting attachment upload") + override var delegate: JobDelegate? = null override var id: String? = null override var failureCount: Int = 0 @@ -46,7 +49,10 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { JobQueue.shared.add(job) } } - if (attachmentsToUpload.isNotEmpty()) return // Wait for all attachments to upload before continuing + if (attachmentsToUpload.isNotEmpty()) { + this.handleFailure(AwaitingUploadException) + return + } // Wait for all attachments to upload before continuing } MessageSender.send(this.message, this.destination).success { this.handleSuccess() diff --git a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt index 30f5d81743..41eb261b95 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt @@ -155,6 +155,11 @@ fun MessageReceiver.handleVisibleMessage(message: VisibleMessage, proto: SignalS val threadID = storage.getOrCreateThreadIdFor(message.syncTarget ?: message.sender!!, message.groupPublicKey, openGroupID) + if (threadID < 0) { + // thread doesn't exist, should only be reached in a case where we are processing open group messages for no longer existent thread + throw MessageReceiver.Error.NoThread + } + val openGroup = threadID.let { storage.getOpenGroup(it.toString()) } @@ -233,7 +238,7 @@ fun MessageReceiver.handleVisibleMessage(message: VisibleMessage, proto: SignalS } val openGroupServerID = message.openGroupServerMessageID if (openGroupServerID != null) { - storage.setOpenGroupServerMessageID(messageID, openGroupServerID, threadID, !(message.isMediaMessage() || attachments.isNotEmpty())) + storage.setOpenGroupServerMessageID(messageID, openGroupServerID, threadID, !message.isMediaMessage()) } // Cancel any typing indicators if needed cancelTypingIndicatorsIfNeeded(message.sender!!) From 26601dbcb208720d040dcc6c2b1c0edaee285a83 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 09:24:13 +1000 Subject: [PATCH 22/31] Clean up background poll worker --- .../loki/api/BackgroundPollWorker.kt | 68 ++++++------------- 1 file changed, 20 insertions(+), 48 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt index a8d00d690c..86970cbcdc 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/api/BackgroundPollWorker.kt @@ -8,7 +8,6 @@ import nl.komponents.kovenant.Promise import nl.komponents.kovenant.all import nl.komponents.kovenant.functional.map import org.session.libsession.messaging.jobs.MessageReceiveJob -import org.session.libsession.messaging.open_groups.OpenGroup import org.session.libsession.messaging.open_groups.OpenGroupV2 import org.session.libsession.messaging.sending_receiving.pollers.ClosedGroupPoller import org.session.libsession.messaging.sending_receiving.pollers.OpenGroupPoller @@ -17,7 +16,6 @@ import org.session.libsession.snode.SnodeAPI import org.session.libsession.utilities.TextSecurePreferences import org.session.libsignal.utilities.logging.Log import org.thoughtcrime.securesms.database.DatabaseFactory -import java.io.IOException import java.util.concurrent.TimeUnit class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Worker(context, params) { @@ -25,45 +23,23 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor companion object { const val TAG = "BackgroundPollWorker" - private const val RETRY_ATTEMPTS = 3 - - @JvmStatic - fun scheduleInstant(context: Context) { - val workRequest = OneTimeWorkRequestBuilder() - .setConstraints(Constraints.Builder() - .setRequiredNetworkType(NetworkType.CONNECTED) - .build() - ) - .build() - - WorkManager - .getInstance(context) - .enqueue(workRequest) - } - @JvmStatic fun schedulePeriodic(context: Context) { Log.v(TAG, "Scheduling periodic work.") - val workRequest = PeriodicWorkRequestBuilder(15, TimeUnit.MINUTES) - .setConstraints(Constraints.Builder() - .setRequiredNetworkType(NetworkType.CONNECTED) - .build() - ) - .build() - - WorkManager - .getInstance(context) - .enqueueUniquePeriodicWork( - TAG, - ExistingPeriodicWorkPolicy.KEEP, - workRequest - ) + val builder = PeriodicWorkRequestBuilder(5, TimeUnit.MINUTES) + builder.setConstraints(Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build()) + val workRequest = builder.build() + WorkManager.getInstance(context).enqueueUniquePeriodicWork( + TAG, + ExistingPeriodicWorkPolicy.REPLACE, + workRequest + ) } } override fun doWork(): Result { if (TextSecurePreferences.getLocalNumber(context) == null) { - Log.v(TAG, "Background poll is canceled due to the Session user is not set up yet.") + Log.v(TAG, "User not registered yet.") return Result.failure() } @@ -71,44 +47,41 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor Log.v(TAG, "Performing background poll.") val promises = mutableListOf>() - // Private chats + // DMs val userPublicKey = TextSecurePreferences.getLocalNumber(context)!! - val privateChatsPromise = SnodeAPI.getMessages(userPublicKey).map { envelopes -> + val dmsPromise = SnodeAPI.getMessages(userPublicKey).map { envelopes -> envelopes.map { envelope -> // FIXME: Using a job here seems like a bad idea... MessageReceiveJob(envelope.toByteArray(), false).executeAsync() } } - promises.addAll(privateChatsPromise.get()) + promises.addAll(dmsPromise.get()) // Closed groups promises.addAll(ClosedGroupPoller().pollOnce()) // Open Groups - val openGroups = DatabaseFactory.getLokiThreadDatabase(context).getAllPublicChats().map { (_,chat)-> - OpenGroup(chat.channel, chat.server, chat.displayName, chat.isDeletable) - } + val openGroups = DatabaseFactory.getLokiThreadDatabase(context).getAllPublicChats().values for (openGroup in openGroups) { val poller = OpenGroupPoller(openGroup) promises.add(poller.pollForNewMessages()) } - val openGroupsV2 = DatabaseFactory.getLokiThreadDatabase(context).getAllV2OpenGroups().values.groupBy(OpenGroupV2::server) + val v2OpenGroups = DatabaseFactory.getLokiThreadDatabase(context).getAllV2OpenGroups().values.groupBy(OpenGroupV2::server) - openGroupsV2.values.map { groups -> + v2OpenGroups.values.map { groups -> OpenGroupV2Poller(groups) }.forEach { poller -> - promises.add(poller.compactPoll(true).map{ /*Unit*/ }) + promises.add(poller.compactPoll(true).map { }) } - // Wait till all the promises get resolved + // Wait until all the promises are resolved all(promises).get() return Result.success() } catch (exception: Exception) { - Log.v(TAG, "Background poll failed due to error: ${exception.message}.", exception) - - return if (runAttemptCount < RETRY_ATTEMPTS) Result.retry() else Result.failure() + Log.e(TAG, "Background poll failed due to error: ${exception.message}.", exception) + return Result.retry() } } @@ -117,8 +90,7 @@ class BackgroundPollWorker(val context: Context, params: WorkerParameters) : Wor override fun onReceive(context: Context, intent: Intent) { if (intent.action == Intent.ACTION_BOOT_COMPLETED) { Log.v(TAG, "Boot broadcast caught.") - BackgroundPollWorker.scheduleInstant(context) - BackgroundPollWorker.schedulePeriodic(context) + schedulePeriodic(context) } } } From 3cab81c329437072dc1308681d55d0e546c0bb26 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 09:38:39 +1000 Subject: [PATCH 23/31] Fix message send job attachment upload handling --- .../securesms/database/Storage.kt | 2 +- .../loki/database/SessionJobDatabase.kt | 8 +++--- .../libsession/messaging/StorageProtocol.kt | 2 +- .../messaging/jobs/AttachmentUploadJob.kt | 2 +- .../libsession/messaging/jobs/JobQueue.kt | 26 ++++++++++++++----- .../messaging/jobs/MessageSendJob.kt | 4 +-- .../ReceivedMessageHandler.kt | 2 +- 7 files changed, 29 insertions(+), 17 deletions(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt index f93670ae88..f1602a8e17 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt @@ -189,7 +189,7 @@ class Storage(context: Context, helper: SQLCipherOpenHelper) : Database(context, DatabaseFactory.getSessionJobDatabase(context).markJobAsSucceeded(jobId) } - override fun markJobAsFailed(jobId: String) { + override fun markJobAsFailedPermanently(jobId: String) { DatabaseFactory.getSessionJobDatabase(context).markJobAsFailed(jobId) } diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt index d8c072dd5f..f684a778aa 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt @@ -25,18 +25,18 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa fun persistJob(job: Job) { val database = databaseHelper.writableDatabase val contentValues = ContentValues(4) - contentValues.put(jobID, job.id) + contentValues.put(jobID, job.id!!) contentValues.put(jobType, job.getFactoryKey()) contentValues.put(failureCount, job.failureCount) contentValues.put(serializedData, SessionJobHelper.dataSerializer.serialize(job.serialize())) - database.insertOrUpdate(sessionJobTable, contentValues, "$jobID = ?", arrayOf(job.id!!)) + database.insertOrUpdate(sessionJobTable, contentValues, "$jobID = ?", arrayOf( job.id!! )) } fun markJobAsSucceeded(jobID: String) { databaseHelper.writableDatabase.delete(sessionJobTable, "${Companion.jobID} = ?", arrayOf( jobID )) } - fun markJobAsFailed(jobID: String) { + fun markJobAsFailedPermanently(jobID: String) { databaseHelper.writableDatabase.delete(sessionJobTable, "${Companion.jobID} = ?", arrayOf( jobID )) } @@ -74,7 +74,7 @@ class SessionJobDatabase(context: Context, helper: SQLCipherOpenHelper) : Databa val database = databaseHelper.readableDatabase var cursor: android.database.Cursor? = null try { - cursor = database.rawQuery("SELECT * FROM $sessionJobTable WHERE $jobID = ?", arrayOf( job.id )) + cursor = database.rawQuery("SELECT * FROM $sessionJobTable WHERE $jobID = ?", arrayOf( job.id!! )) return cursor == null || !cursor.moveToFirst() } catch (e: Exception) { // Do nothing diff --git a/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt b/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt index da604264d1..d850e30664 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/StorageProtocol.kt @@ -45,7 +45,7 @@ interface StorageProtocol { // Jobs fun persistJob(job: Job) fun markJobAsSucceeded(jobId: String) - fun markJobAsFailed(jobId: String) + fun markJobAsFailedPermanently(jobId: String) fun getAllPendingJobs(type: String): Map fun getAttachmentUploadJob(attachmentID: Long): AttachmentUploadJob? fun getMessageSendJob(messageSendJobID: String): MessageSendJob? diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt index cbdcd42fca..690caf512c 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt @@ -103,7 +103,7 @@ class AttachmentUploadJob(val attachmentID: Long, val threadID: String, val mess val messageSendJob = storage.getMessageSendJob(messageSendJobID) MessageSender.handleFailedMessageSend(this.message, e) if (messageSendJob != null) { - storage.markJobAsFailed(messageSendJobID) + storage.markJobAsFailedPermanently(messageSendJobID) } } diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt index fab49384fd..a2f47556bc 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/JobQueue.kt @@ -5,6 +5,7 @@ import kotlinx.coroutines.channels.Channel import kotlinx.coroutines.channels.Channel.Factory.UNLIMITED import org.session.libsession.messaging.MessagingModuleConfiguration import org.session.libsignal.utilities.logging.Log +import java.lang.IllegalStateException import java.util.* import java.util.concurrent.ConcurrentHashMap import java.util.concurrent.Executors @@ -47,16 +48,15 @@ class JobQueue : JobDelegate { while (isActive) { for (job in queue) { when (job) { - is NotifyPNServerJob, - is AttachmentUploadJob, - is MessageSendJob -> txQueue.send(job) + is NotifyPNServerJob, is AttachmentUploadJob, is MessageSendJob -> txQueue.send(job) is AttachmentDownloadJob -> attachmentQueue.send(job) - else -> rxQueue.send(job) + is MessageReceiveJob -> rxQueue.send(job) + else -> throw IllegalStateException("Unexpected job type.") } } } - // job has been cancelled + // The job has been cancelled receiveJob.cancel() txJob.cancel() attachmentJob.cancel() @@ -123,11 +123,23 @@ class JobQueue : JobDelegate { } override fun handleJobFailed(job: Job, error: Exception) { - job.failureCount += 1 + // Canceled val storage = MessagingModuleConfiguration.shared.storage if (storage.isJobCanceled(job)) { return Log.i("Loki", "${job::class.simpleName} canceled.") } + // Message send jobs waiting for the attachment to upload + if (job is MessageSendJob && error is MessageSendJob.AwaitingAttachmentUploadException) { + val retryInterval: Long = 1000 * 4 + Log.i("Loki", "Message send job waiting for attachment upload to finish.") + timer.schedule(delay = retryInterval) { + Log.i("Loki", "Retrying ${job::class.simpleName}.") + queue.offer(job) + } + return + } + // Regular job failure + job.failureCount += 1 if (job.failureCount >= job.maxFailureCount) { handleJobFailedPermanently(job, error) } else { @@ -148,7 +160,7 @@ class JobQueue : JobDelegate { private fun handleJobFailedPermanently(jobId: String) { val storage = MessagingModuleConfiguration.shared.storage - storage.markJobAsFailed(jobId) + storage.markJobAsFailedPermanently(jobId) } private fun getRetryInterval(job: Job): Long { diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index 1a0e4e57f4..b93aa13dc2 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -13,7 +13,7 @@ import org.session.libsignal.utilities.logging.Log class MessageSendJob(val message: Message, val destination: Destination) : Job { - object AwaitingUploadException: Exception("Awaiting attachment upload") + object AwaitingAttachmentUploadException : Exception("Awaiting attachment upload.") override var delegate: JobDelegate? = null override var id: String? = null @@ -50,7 +50,7 @@ class MessageSendJob(val message: Message, val destination: Destination) : Job { } } if (attachmentsToUpload.isNotEmpty()) { - this.handleFailure(AwaitingUploadException) + this.handleFailure(AwaitingAttachmentUploadException) return } // Wait for all attachments to upload before continuing } diff --git a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt index 41eb261b95..be331891b0 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/sending_receiving/ReceivedMessageHandler.kt @@ -153,7 +153,7 @@ fun MessageReceiver.handleVisibleMessage(message: VisibleMessage, proto: SignalS // Get or create thread val threadID = storage.getOrCreateThreadIdFor(message.syncTarget - ?: message.sender!!, message.groupPublicKey, openGroupID) + ?: message.sender!!, message.groupPublicKey, openGroupID) if (threadID < 0) { // thread doesn't exist, should only be reached in a case where we are processing open group messages for no longer existent thread From 43ba8299776ba25efca376a592a1ce904064b5bc Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 09:40:07 +1000 Subject: [PATCH 24/31] Fix build --- .../main/java/org/thoughtcrime/securesms/database/Storage.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt index f1602a8e17..2c82d60aa7 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/database/Storage.kt @@ -190,7 +190,7 @@ class Storage(context: Context, helper: SQLCipherOpenHelper) : Database(context, } override fun markJobAsFailedPermanently(jobId: String) { - DatabaseFactory.getSessionJobDatabase(context).markJobAsFailed(jobId) + DatabaseFactory.getSessionJobDatabase(context).markJobAsFailedPermanently(jobId) } override fun getAllPendingJobs(type: String): Map { From af84b1ef3a97ffb64779940d3ffc1ac864dd4c22 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 09:45:29 +1000 Subject: [PATCH 25/31] Update build number --- app/build.gradle | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/build.gradle b/app/build.gradle index 74738b8801..dc0e327af9 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -158,7 +158,7 @@ dependencies { testImplementation 'org.robolectric:shadows-multidex:4.2' } -def canonicalVersionCode = 159 +def canonicalVersionCode = 161 def canonicalVersionName = "1.10.2" def postFixSize = 10 From 115bc9b159bcae12a2d9b6b2256422c9a19ed453 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 10:31:06 +1000 Subject: [PATCH 26/31] Speed up path building --- app/build.gradle | 2 +- .../securesms/jobmanager/Job.java | 2 +- .../securesms/jobmanager/JobController.java | 2 +- .../securesms/jobmanager/JobInstantiator.java | 2 +- .../securesms/jobmanager/JobManager.java | 2 +- .../jobmanager/impl/JsonDataSerializer.java | 2 +- .../securesms/jobs/AvatarDownloadJob.java | 2 +- .../securesms/jobs/LocalBackupJob.java | 2 +- .../jobs/RetrieveProfileAvatarJob.java | 2 +- .../securesms/jobs/TrimThreadJob.java | 2 +- .../securesms/jobs/UpdateApkJob.java | 2 +- .../api/PrepareAttachmentAudioExtrasJob.kt | 2 +- .../loki/database/SessionJobDatabase.kt | 1 + .../impl/JsonDataSerializerTest.java | 2 +- build.gradle | 2 +- .../messaging/jobs/AttachmentDownloadJob.kt | 1 + .../messaging/jobs/AttachmentUploadJob.kt | 1 + .../session/libsession/messaging/jobs/Job.kt | 2 + .../messaging/jobs/MessageReceiveJob.kt | 1 + .../messaging/jobs/MessageSendJob.kt | 1 + .../messaging/jobs/NotifyPNServerJob.kt | 1 + .../messaging/jobs/SessionJobInstantiator.kt | 2 + .../messaging/{jobs => utilities}/Data.java | 85 +++++++++++-------- .../libsession/snode/OnionRequestAPI.kt | 7 +- .../session/libsignal/service/loki/HTTP.kt | 34 +++++--- 25 files changed, 100 insertions(+), 64 deletions(-) rename libsession/src/main/java/org/session/libsession/messaging/{jobs => utilities}/Data.java (85%) diff --git a/app/build.gradle b/app/build.gradle index dc0e327af9..8b453260bb 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -8,7 +8,7 @@ buildscript { jcenter() } dependencies { - classpath 'com.android.tools.build:gradle:4.1.2' + classpath 'com.android.tools.build:gradle:4.1.3' classpath files('libs/gradle-witness.jar') classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" classpath "org.jetbrains.kotlin:kotlin-serialization:$kotlin_version" diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/Job.java b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/Job.java index 23a7b22e9f..d82e851f39 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/Job.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/Job.java @@ -5,7 +5,7 @@ import androidx.annotation.NonNull; import androidx.annotation.Nullable; import androidx.annotation.WorkerThread; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.session.libsignal.utilities.logging.Log; import java.util.LinkedList; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobController.java b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobController.java index d0f99ce3f3..17be996b3c 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobController.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobController.java @@ -7,7 +7,7 @@ import androidx.annotation.WorkerThread; import com.annimon.stream.Stream; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.thoughtcrime.securesms.jobmanager.persistence.ConstraintSpec; import org.thoughtcrime.securesms.jobmanager.persistence.DependencySpec; import org.thoughtcrime.securesms.jobmanager.persistence.FullSpec; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobInstantiator.java b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobInstantiator.java index c50e1dc207..6d1527d131 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobInstantiator.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobInstantiator.java @@ -2,7 +2,7 @@ package org.thoughtcrime.securesms.jobmanager; import androidx.annotation.NonNull; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import java.util.HashMap; import java.util.Map; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobManager.java b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobManager.java index 73fb63dcd2..6b101faed2 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobManager.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/JobManager.java @@ -5,7 +5,7 @@ import android.content.Intent; import android.os.Build; import androidx.annotation.NonNull; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.thoughtcrime.securesms.jobmanager.impl.DefaultExecutorFactory; import org.thoughtcrime.securesms.jobmanager.impl.JsonDataSerializer; import org.thoughtcrime.securesms.jobmanager.persistence.JobStorage; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializer.java b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializer.java index e3b5b77e64..87854452a5 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializer.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializer.java @@ -2,7 +2,7 @@ package org.thoughtcrime.securesms.jobmanager.impl; import androidx.annotation.NonNull; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.session.libsignal.utilities.logging.Log; import org.session.libsignal.utilities.JsonUtil; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobs/AvatarDownloadJob.java b/app/src/main/java/org/thoughtcrime/securesms/jobs/AvatarDownloadJob.java index 34faaf48a4..e79db7d00e 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobs/AvatarDownloadJob.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobs/AvatarDownloadJob.java @@ -3,7 +3,7 @@ package org.thoughtcrime.securesms.jobs; import android.graphics.Bitmap; import androidx.annotation.NonNull; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.session.libsession.utilities.DownloadUtilities; import org.session.libsignal.service.api.crypto.AttachmentCipherInputStream; import org.thoughtcrime.securesms.database.DatabaseFactory; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobs/LocalBackupJob.java b/app/src/main/java/org/thoughtcrime/securesms/jobs/LocalBackupJob.java index d861fb267a..f8e0c531f7 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobs/LocalBackupJob.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobs/LocalBackupJob.java @@ -2,7 +2,7 @@ package org.thoughtcrime.securesms.jobs; import androidx.annotation.NonNull; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.session.libsignal.utilities.externalstorage.NoExternalStorageException; import org.thoughtcrime.securesms.jobmanager.Job; import org.session.libsignal.utilities.logging.Log; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobs/RetrieveProfileAvatarJob.java b/app/src/main/java/org/thoughtcrime/securesms/jobs/RetrieveProfileAvatarJob.java index 8211032292..a9c2025258 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobs/RetrieveProfileAvatarJob.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobs/RetrieveProfileAvatarJob.java @@ -7,7 +7,7 @@ import android.text.TextUtils; import androidx.annotation.NonNull; import org.session.libsession.messaging.avatars.AvatarHelper; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.session.libsession.messaging.threads.Address; import org.session.libsession.messaging.threads.recipients.Recipient; import org.session.libsession.utilities.DownloadUtilities; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobs/TrimThreadJob.java b/app/src/main/java/org/thoughtcrime/securesms/jobs/TrimThreadJob.java index 3b7444d58a..bbd51c8837 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobs/TrimThreadJob.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobs/TrimThreadJob.java @@ -18,7 +18,7 @@ package org.thoughtcrime.securesms.jobs; import androidx.annotation.NonNull; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.thoughtcrime.securesms.database.DatabaseFactory; import org.thoughtcrime.securesms.jobmanager.Job; import org.session.libsignal.utilities.logging.Log; diff --git a/app/src/main/java/org/thoughtcrime/securesms/jobs/UpdateApkJob.java b/app/src/main/java/org/thoughtcrime/securesms/jobs/UpdateApkJob.java index 4354b83961..81e34b5a58 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/jobs/UpdateApkJob.java +++ b/app/src/main/java/org/thoughtcrime/securesms/jobs/UpdateApkJob.java @@ -13,7 +13,7 @@ import androidx.annotation.Nullable; import com.fasterxml.jackson.annotation.JsonProperty; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.thoughtcrime.securesms.jobmanager.Job; import org.thoughtcrime.securesms.jobmanager.impl.NetworkConstraint; import org.session.libsignal.utilities.logging.Log; diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/api/PrepareAttachmentAudioExtrasJob.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/api/PrepareAttachmentAudioExtrasJob.kt index c659146caf..dc97760758 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/api/PrepareAttachmentAudioExtrasJob.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/api/PrepareAttachmentAudioExtrasJob.kt @@ -5,7 +5,7 @@ import android.os.Build import org.session.libsignal.utilities.logging.Log import androidx.annotation.RequiresApi import org.greenrobot.eventbus.EventBus -import org.session.libsession.messaging.jobs.Data +import org.session.libsession.messaging.utilities.Data import org.session.libsession.messaging.sending_receiving.attachments.Attachment import org.session.libsession.messaging.sending_receiving.attachments.AttachmentId import org.session.libsession.messaging.sending_receiving.attachments.DatabaseAttachmentAudioExtras diff --git a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt index f684a778aa..22c8d48f44 100644 --- a/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt +++ b/app/src/main/java/org/thoughtcrime/securesms/loki/database/SessionJobDatabase.kt @@ -4,6 +4,7 @@ import android.content.ContentValues import android.content.Context import net.sqlcipher.Cursor import org.session.libsession.messaging.jobs.* +import org.session.libsession.messaging.utilities.Data import org.session.libsignal.utilities.logging.Log import org.thoughtcrime.securesms.database.Database import org.thoughtcrime.securesms.database.helpers.SQLCipherOpenHelper diff --git a/app/src/test/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializerTest.java b/app/src/test/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializerTest.java index 8b2c8ff4fb..04d90ec21c 100644 --- a/app/src/test/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializerTest.java +++ b/app/src/test/java/org/thoughtcrime/securesms/jobmanager/impl/JsonDataSerializerTest.java @@ -1,7 +1,7 @@ package org.thoughtcrime.securesms.jobmanager.impl; import org.junit.Test; -import org.session.libsession.messaging.jobs.Data; +import org.session.libsession.messaging.utilities.Data; import org.session.libsession.utilities.Util; import java.io.IOException; diff --git a/build.gradle b/build.gradle index 2730f69109..391d1279dc 100644 --- a/build.gradle +++ b/build.gradle @@ -6,7 +6,7 @@ buildscript { jcenter() } dependencies { - classpath 'com.android.tools.build:gradle:4.1.2' + classpath 'com.android.tools.build:gradle:4.1.3' classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion" classpath "com.google.gms:google-services:4.3.4" classpath files('libs/gradle-witness.jar') diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt index 8285813206..a08ddb911d 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentDownloadJob.kt @@ -5,6 +5,7 @@ import org.session.libsession.messaging.MessagingModuleConfiguration import org.session.libsession.messaging.file_server.FileServerAPI import org.session.libsession.messaging.open_groups.OpenGroupAPIV2 import org.session.libsession.messaging.sending_receiving.attachments.AttachmentState +import org.session.libsession.messaging.utilities.Data import org.session.libsession.messaging.utilities.DotNetAPI import org.session.libsession.utilities.DownloadUtilities import org.session.libsignal.service.api.crypto.AttachmentCipherInputStream diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt index 690caf512c..a4ef41431d 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/AttachmentUploadJob.kt @@ -8,6 +8,7 @@ import org.session.libsession.messaging.file_server.FileServerAPI import org.session.libsession.messaging.messages.Message import org.session.libsession.messaging.open_groups.OpenGroupAPIV2 import org.session.libsession.messaging.sending_receiving.MessageSender +import org.session.libsession.messaging.utilities.Data import org.session.libsession.messaging.utilities.DotNetAPI import org.session.libsignal.service.api.crypto.AttachmentCipherOutputStream import org.session.libsignal.service.api.messages.SignalServiceAttachmentStream diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt index ca2c8c9629..74feb83a61 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/Job.kt @@ -1,5 +1,7 @@ package org.session.libsession.messaging.jobs +import org.session.libsession.messaging.utilities.Data + interface Job { var delegate: JobDelegate? var id: String? diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt index ebca416250..256091ada4 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageReceiveJob.kt @@ -4,6 +4,7 @@ import nl.komponents.kovenant.Promise import nl.komponents.kovenant.deferred import org.session.libsession.messaging.sending_receiving.MessageReceiver import org.session.libsession.messaging.sending_receiving.handle +import org.session.libsession.messaging.utilities.Data import org.session.libsignal.utilities.logging.Log class MessageReceiveJob(val data: ByteArray, val isBackgroundPoll: Boolean, val openGroupMessageServerID: Long? = null, val openGroupID: String? = null) : Job { diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt index b93aa13dc2..2989155314 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/MessageSendJob.kt @@ -9,6 +9,7 @@ import org.session.libsession.messaging.messages.Destination import org.session.libsession.messaging.messages.Message import org.session.libsession.messaging.messages.visible.VisibleMessage import org.session.libsession.messaging.sending_receiving.MessageSender +import org.session.libsession.messaging.utilities.Data import org.session.libsignal.utilities.logging.Log class MessageSendJob(val message: Message, val destination: Destination) : Job { diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt index 8aa69a5859..0445eaf8f5 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/NotifyPNServerJob.kt @@ -9,6 +9,7 @@ import okhttp3.Request import okhttp3.RequestBody import org.session.libsession.messaging.sending_receiving.notifications.PushNotificationAPI +import org.session.libsession.messaging.utilities.Data import org.session.libsession.snode.SnodeMessage import org.session.libsession.snode.OnionRequestAPI diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt b/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt index a6336e9148..311448578d 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt +++ b/libsession/src/main/java/org/session/libsession/messaging/jobs/SessionJobInstantiator.kt @@ -1,5 +1,7 @@ package org.session.libsession.messaging.jobs +import org.session.libsession.messaging.utilities.Data + class SessionJobInstantiator(private val jobFactories: Map>) { fun instantiate(jobFactoryKey: String, data: Data): Job? { diff --git a/libsession/src/main/java/org/session/libsession/messaging/jobs/Data.java b/libsession/src/main/java/org/session/libsession/messaging/utilities/Data.java similarity index 85% rename from libsession/src/main/java/org/session/libsession/messaging/jobs/Data.java rename to libsession/src/main/java/org/session/libsession/messaging/utilities/Data.java index 310cfed336..c3502d62cb 100644 --- a/libsession/src/main/java/org/session/libsession/messaging/jobs/Data.java +++ b/libsession/src/main/java/org/session/libsession/messaging/utilities/Data.java @@ -1,4 +1,4 @@ -package org.session.libsession.messaging.jobs; +package org.session.libsession.messaging.utilities; import android.os.Parcelable; @@ -12,11 +12,7 @@ import org.session.libsession.utilities.ParcelableUtil; import java.util.HashMap; import java.util.Map; -// Introduce a dedicated Map field specifically for parcelable needs. public class Data { - - public static final Data EMPTY = new Data.Builder().build(); - @JsonProperty private final Map strings; @JsonProperty private final Map stringArrays; @JsonProperty private final Map integers; @@ -31,20 +27,23 @@ public class Data { @JsonProperty private final Map booleanArrays; @JsonProperty private final Map byteArrays; - public Data(@JsonProperty("strings") @NonNull Map strings, - @JsonProperty("stringArrays") @NonNull Map stringArrays, - @JsonProperty("integers") @NonNull Map integers, - @JsonProperty("integerArrays") @NonNull Map integerArrays, - @JsonProperty("longs") @NonNull Map longs, - @JsonProperty("longArrays") @NonNull Map longArrays, - @JsonProperty("floats") @NonNull Map floats, - @JsonProperty("floatArrays") @NonNull Map floatArrays, - @JsonProperty("doubles") @NonNull Map doubles, - @JsonProperty("doubleArrays") @NonNull Map doubleArrays, - @JsonProperty("booleans") @NonNull Map booleans, - @JsonProperty("booleanArrays") @NonNull Map booleanArrays, - @JsonProperty("byteArrays") @NonNull Map byteArrays) - { + public static final Data EMPTY = new Data.Builder().build(); + + public Data( + @JsonProperty("strings") @NonNull Map strings, + @JsonProperty("stringArrays") @NonNull Map stringArrays, + @JsonProperty("integers") @NonNull Map integers, + @JsonProperty("integerArrays") @NonNull Map integerArrays, + @JsonProperty("longs") @NonNull Map longs, + @JsonProperty("longArrays") @NonNull Map longArrays, + @JsonProperty("floats") @NonNull Map floats, + @JsonProperty("floatArrays") @NonNull Map floatArrays, + @JsonProperty("doubles") @NonNull Map doubles, + @JsonProperty("doubleArrays") @NonNull Map doubleArrays, + @JsonProperty("booleans") @NonNull Map booleans, + @JsonProperty("booleanArrays") @NonNull Map booleanArrays, + @JsonProperty("byteArrays") @NonNull Map byteArrays + ) { this.strings = strings; this.stringArrays = stringArrays; this.integers = integers; @@ -75,6 +74,7 @@ public class Data { } + public boolean hasStringArray(@NonNull String key) { return stringArrays.containsKey(key); } @@ -100,6 +100,7 @@ public class Data { } + public boolean hasIntegerArray(@NonNull String key) { return integerArrays.containsKey(key); } @@ -110,6 +111,7 @@ public class Data { } + public boolean hasLong(@NonNull String key) { return longs.containsKey(key); } @@ -125,6 +127,7 @@ public class Data { } + public boolean hasLongArray(@NonNull String key) { return longArrays.containsKey(key); } @@ -135,6 +138,7 @@ public class Data { } + public boolean hasFloat(@NonNull String key) { return floats.containsKey(key); } @@ -150,6 +154,7 @@ public class Data { } + public boolean hasFloatArray(@NonNull String key) { return floatArrays.containsKey(key); } @@ -160,6 +165,7 @@ public class Data { } + public boolean hasDouble(@NonNull String key) { return doubles.containsKey(key); } @@ -175,6 +181,7 @@ public class Data { } + public boolean hasDoubleArray(@NonNull String key) { return floatArrays.containsKey(key); } @@ -185,6 +192,7 @@ public class Data { } + public boolean hasBoolean(@NonNull String key) { return booleans.containsKey(key); } @@ -200,6 +208,7 @@ public class Data { } + public boolean hasBooleanArray(@NonNull String key) { return booleanArrays.containsKey(key); } @@ -209,6 +218,8 @@ public class Data { return booleanArrays.get(key); } + + public boolean hasByteArray(@NonNull String key) { return byteArrays.containsKey(key); } @@ -218,6 +229,8 @@ public class Data { return byteArrays.get(key); } + + public boolean hasParcelable(@NonNull String key) { return byteArrays.containsKey(key); } @@ -228,6 +241,8 @@ public class Data { return ParcelableUtil.unmarshall(bytes, creator); } + + private void throwIfAbsent(@NonNull Map map, @NonNull String key) { if (!map.containsKey(key)) { throw new IllegalStateException("Tried to retrieve a value with key '" + key + "', but it wasn't present."); @@ -236,7 +251,6 @@ public class Data { public static class Builder { - private final Map strings = new HashMap<>(); private final Map stringArrays = new HashMap<>(); private final Map integers = new HashMap<>(); @@ -323,19 +337,21 @@ public class Data { } public Data build() { - return new Data(strings, - stringArrays, - integers, - integerArrays, - longs, - longArrays, - floats, - floatArrays, - doubles, - doubleArrays, - booleans, - booleanArrays, - byteArrays); + return new Data( + strings, + stringArrays, + integers, + integerArrays, + longs, + longArrays, + floats, + floatArrays, + doubles, + doubleArrays, + booleans, + booleanArrays, + byteArrays + ); } } @@ -343,5 +359,4 @@ public class Data { @NonNull String serialize(@NonNull Data data); @NonNull Data deserialize(@NonNull String serialized); } -} - +} \ No newline at end of file diff --git a/libsession/src/main/java/org/session/libsession/snode/OnionRequestAPI.kt b/libsession/src/main/java/org/session/libsession/snode/OnionRequestAPI.kt index 086b911933..c3e454e6bc 100644 --- a/libsession/src/main/java/org/session/libsession/snode/OnionRequestAPI.kt +++ b/libsession/src/main/java/org/session/libsession/snode/OnionRequestAPI.kt @@ -53,11 +53,11 @@ object OnionRequestAPI { /** * The number of times a path can fail before it's replaced. */ - private const val pathFailureThreshold = 1 + private const val pathFailureThreshold = 3 /** * The number of times a snode can fail before it's replaced. */ - private const val snodeFailureThreshold = 1 + private const val snodeFailureThreshold = 3 /** * The number of guard snodes required to maintain `targetPathCount` paths. */ @@ -93,7 +93,7 @@ object OnionRequestAPI { ThreadUtils.queue { // No need to block the shared context for this val url = "${snode.address}:${snode.port}/get_stats/v1" try { - val json = HTTP.execute(HTTP.Verb.GET, url) + val json = HTTP.execute(HTTP.Verb.GET, url, 3) val version = json["version"] as? String if (version == null) { deferred.reject(Exception("Missing snode version.")); return@queue } if (version >= "2.0.7") { @@ -463,7 +463,6 @@ object OnionRequestAPI { "method" to request.method(), "headers" to headers ) - url.isHttps val destination = Destination.Server(host, target, x25519PublicKey, url.scheme(), url.port()) return sendOnionRequest(destination, payload, isJSONRequired).recover { exception -> Log.d("Loki", "Couldn't reach server: $urlAsString due to error: $exception.") diff --git a/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt b/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt index 11e7ed6154..774e17cd8d 100644 --- a/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt +++ b/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt @@ -3,6 +3,7 @@ package org.session.libsignal.service.loki import okhttp3.* import org.session.libsignal.utilities.logging.Log import org.session.libsignal.utilities.JsonUtil +import java.lang.IllegalStateException import java.security.SecureRandom import java.security.cert.X509Certificate import java.util.concurrent.TimeUnit @@ -25,9 +26,7 @@ object HTTP { override fun checkClientTrusted(chain: Array?, authorizationType: String?) { } override fun checkServerTrusted(chain: Array?, authorizationType: String?) { } - override fun getAcceptedIssuers(): Array { - return arrayOf() - } + override fun getAcceptedIssuers(): Array { return arrayOf() } } val sslContext = SSLContext.getInstance("SSL") sslContext.init(null, arrayOf( trustManager ), SecureRandom()) @@ -40,7 +39,7 @@ object HTTP { .build() } - private const val timeout: Long = 20 + private const val timeout: Long = 10 class HTTPRequestFailedException(val statusCode: Int, val json: Map<*, *>?) : kotlin.Exception("HTTP request failed with status code $statusCode.") @@ -52,26 +51,26 @@ object HTTP { /** * Sync. Don't call from the main thread. */ - fun execute(verb: Verb, url: String, useSeedNodeConnection: Boolean = false): Map<*, *> { - return execute(verb = verb, url = url, body = null, useSeedNodeConnection = useSeedNodeConnection) + fun execute(verb: Verb, url: String, timeout: Long = HTTP.timeout, useSeedNodeConnection: Boolean = false): Map<*, *> { + return execute(verb = verb, url = url, body = null, timeout = timeout, useSeedNodeConnection = useSeedNodeConnection) } /** * Sync. Don't call from the main thread. */ - fun execute(verb: Verb, url: String, parameters: Map?, useSeedNodeConnection: Boolean = false): Map<*, *> { + fun execute(verb: Verb, url: String, parameters: Map?, timeout: Long = HTTP.timeout, useSeedNodeConnection: Boolean = false): Map<*, *> { if (parameters != null) { val body = JsonUtil.toJson(parameters).toByteArray() - return execute(verb = verb, url = url, body = body, useSeedNodeConnection = useSeedNodeConnection) + return execute(verb = verb, url = url, body = body, timeout = timeout, useSeedNodeConnection = useSeedNodeConnection) } else { - return execute(verb = verb, url = url, body = null, useSeedNodeConnection = useSeedNodeConnection) + return execute(verb = verb, url = url, body = null, timeout = timeout, useSeedNodeConnection = useSeedNodeConnection) } } /** * Sync. Don't call from the main thread. */ - fun execute(verb: Verb, url: String, body: ByteArray?, useSeedNodeConnection: Boolean = false): Map<*, *> { + fun execute(verb: Verb, url: String, body: ByteArray?, timeout: Long = HTTP.timeout, useSeedNodeConnection: Boolean = false): Map<*, *> { val request = Request.Builder().url(url) when (verb) { Verb.GET -> request.get() @@ -85,7 +84,20 @@ object HTTP { } lateinit var response: Response try { - val connection = if (useSeedNodeConnection) seedNodeConnection else defaultConnection + val connection: OkHttpClient + if (timeout != HTTP.timeout) { // Custom timeout + if (useSeedNodeConnection) { + throw IllegalStateException("Setting a custom timeout is only allowed for requests to snodes.") + } + connection = OkHttpClient() + .newBuilder() + .connectTimeout(timeout, TimeUnit.SECONDS) + .readTimeout(timeout, TimeUnit.SECONDS) + .writeTimeout(timeout, TimeUnit.SECONDS) + .build() + } else { + connection = if (useSeedNodeConnection) seedNodeConnection else defaultConnection + } response = connection.newCall(request.build()).execute() } catch (exception: Exception) { Log.d("Loki", "${verb.rawValue} request to $url failed due to error: ${exception.localizedMessage}.") From 288d76d29212c5aa0e6e3442ec9a66f25f51e99c Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 10:33:34 +1000 Subject: [PATCH 27/31] Add documentation --- .../snode/OnionRequestEncryption.kt | 10 +++---- .../session/libsession/snode/SnodeMessage.kt | 26 +++++++++++++------ 2 files changed, 23 insertions(+), 13 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/snode/OnionRequestEncryption.kt b/libsession/src/main/java/org/session/libsession/snode/OnionRequestEncryption.kt index c462e8f94f..d4f19a678b 100644 --- a/libsession/src/main/java/org/session/libsession/snode/OnionRequestEncryption.kt +++ b/libsession/src/main/java/org/session/libsession/snode/OnionRequestEncryption.kt @@ -71,11 +71,11 @@ object OnionRequestEncryption { } is OnionRequestAPI.Destination.Server -> { payload = mutableMapOf( - "host" to rhs.host, - "target" to rhs.target, - "method" to "POST", - "protocol" to rhs.scheme, - "port" to rhs.port + "host" to rhs.host, + "target" to rhs.target, + "method" to "POST", + "protocol" to rhs.scheme, + "port" to rhs.port ) } } diff --git a/libsession/src/main/java/org/session/libsession/snode/SnodeMessage.kt b/libsession/src/main/java/org/session/libsession/snode/SnodeMessage.kt index aa7fd4f6bb..b508cf0ef9 100644 --- a/libsession/src/main/java/org/session/libsession/snode/SnodeMessage.kt +++ b/libsession/src/main/java/org/session/libsession/snode/SnodeMessage.kt @@ -3,23 +3,33 @@ package org.session.libsession.snode import org.session.libsignal.service.loki.utilities.removing05PrefixIfNeeded data class SnodeMessage( - // The hex encoded public key of the recipient. + /** + * The hex encoded public key of the recipient. + */ val recipient: String, - // The content of the message. + /** + * The content of the message. + */ val data: String, - // The time to live for the message in milliseconds. + /** + * The time to live for the message in milliseconds. + */ val ttl: Long, - // When the proof of work was calculated. + /** + * When the proof of work was calculated. + * + * **Note:** Expressed as milliseconds since 00:00:00 UTC on 1 January 1970. + */ val timestamp: Long ) { internal fun toJSON(): Map { return mapOf( "pubKey" to if (SnodeAPI.useTestnet) recipient.removing05PrefixIfNeeded() else recipient, - "data" to data, - "ttl" to ttl.toString(), - "timestamp" to timestamp.toString(), - "nonce" to "" + "data" to data, + "ttl" to ttl.toString(), + "timestamp" to timestamp.toString(), + "nonce" to "" ) } } From b798f49512f2f576a9744841e958faefb456e18f Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 10:38:13 +1000 Subject: [PATCH 28/31] Minor performance optimization --- .../java/org/session/libsession/snode/SnodeAPI.kt | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt index a5094a63ee..586193b7c1 100644 --- a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt +++ b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt @@ -34,7 +34,7 @@ object SnodeAPI { // Settings private val maxRetryCount = 6 private val minimumSnodePoolCount = 12 - private val minimumSwarmSnodeCount = 2 + private val minimumSwarmSnodeCount = 3 // Use port 4433 if the API level can handle the network security configuration and enforce pinned certificates private val seedNodePort = if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) 443 else 4433 private val seedNodePool by lazy { @@ -44,7 +44,7 @@ object SnodeAPI { setOf( "https://storage.seed1.loki.network:$seedNodePort ", "https://storage.seed3.loki.network:$seedNodePort ", "https://public.loki.foundation:$seedNodePort" ) } } - private val snodeFailureThreshold = 4 + private val snodeFailureThreshold = 3 private val targetSwarmSnodeCount = 2 private val useOnionRequests = true @@ -252,19 +252,20 @@ object SnodeAPI { private fun removeDuplicates(publicKey: String, rawMessages: List<*>): List<*> { val receivedMessageHashValues = database.getReceivedMessageHashValues(publicKey)?.toMutableSet() ?: mutableSetOf() - return rawMessages.filter { rawMessage -> + val result = rawMessages.filter { rawMessage -> val rawMessageAsJSON = rawMessage as? Map<*, *> val hashValue = rawMessageAsJSON?.get("hash") as? String if (hashValue != null) { val isDuplicate = receivedMessageHashValues.contains(hashValue) receivedMessageHashValues.add(hashValue) - database.setReceivedMessageHashValues(publicKey, receivedMessageHashValues) !isDuplicate } else { Log.d("Loki", "Missing hash value for message: ${rawMessage?.prettifiedDescription()}.") false } } + database.setReceivedMessageHashValues(publicKey, receivedMessageHashValues) + return result } private fun parseEnvelopes(rawMessages: List<*>): List { @@ -305,7 +306,7 @@ object SnodeAPI { } } when (statusCode) { - 400, 500, 503 -> { // Usually indicates that the snode isn't up to date + 400, 500, 502, 503 -> { // Usually indicates that the snode isn't up to date handleBadSnode() } 406 -> { From 75ce0f056cb3c87fbd285c730e4b9f6e5f2df286 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Thu, 13 May 2021 10:42:53 +1000 Subject: [PATCH 29/31] Use snodes returned in 421 response --- .../org/session/libsession/snode/SnodeAPI.kt | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt index 586193b7c1..ab73604385 100644 --- a/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt +++ b/libsession/src/main/java/org/session/libsession/snode/SnodeAPI.kt @@ -317,8 +317,20 @@ object SnodeAPI { 421 -> { // The snode isn't associated with the given public key anymore if (publicKey != null) { - Log.d("Loki", "Invalidating swarm for: $publicKey.") - dropSnodeFromSwarmIfNeeded(snode, publicKey) + fun invalidateSwarm() { + Log.d("Loki", "Invalidating swarm for: $publicKey.") + dropSnodeFromSwarmIfNeeded(snode, publicKey) + } + if (json != null) { + val snodes = parseSnodes(json) + if (snodes.isNotEmpty()) { + database.setSwarm(publicKey, snodes.toSet()) + } else { + invalidateSwarm() + } + } else { + invalidateSwarm() + } } else { Log.d("Loki", "Got a 421 without an associated public key.") } From 649bfee647bcbf05f0fceda58b30afb271bd4046 Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Fri, 14 May 2021 08:56:08 +1000 Subject: [PATCH 30/31] Fix HTTP utility --- .../session/libsignal/service/loki/HTTP.kt | 26 ++++++++++++++----- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt b/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt index 774e17cd8d..90a20d2282 100644 --- a/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt +++ b/libsignal/src/main/java/org/session/libsignal/service/loki/HTTP.kt @@ -39,6 +39,25 @@ object HTTP { .build() } + private fun getDefaultConnection(timeout: Long): OkHttpClient { + // Snode to snode communication uses self-signed certificates but clients can safely ignore this + val trustManager = object : X509TrustManager { + + override fun checkClientTrusted(chain: Array?, authorizationType: String?) { } + override fun checkServerTrusted(chain: Array?, authorizationType: String?) { } + override fun getAcceptedIssuers(): Array { return arrayOf() } + } + val sslContext = SSLContext.getInstance("SSL") + sslContext.init(null, arrayOf( trustManager ), SecureRandom()) + return OkHttpClient().newBuilder() + .sslSocketFactory(sslContext.socketFactory, trustManager) + .hostnameVerifier { _, _ -> true } + .connectTimeout(timeout, TimeUnit.SECONDS) + .readTimeout(timeout, TimeUnit.SECONDS) + .writeTimeout(timeout, TimeUnit.SECONDS) + .build() + } + private const val timeout: Long = 10 class HTTPRequestFailedException(val statusCode: Int, val json: Map<*, *>?) @@ -89,12 +108,7 @@ object HTTP { if (useSeedNodeConnection) { throw IllegalStateException("Setting a custom timeout is only allowed for requests to snodes.") } - connection = OkHttpClient() - .newBuilder() - .connectTimeout(timeout, TimeUnit.SECONDS) - .readTimeout(timeout, TimeUnit.SECONDS) - .writeTimeout(timeout, TimeUnit.SECONDS) - .build() + connection = getDefaultConnection(timeout) } else { connection = if (useSeedNodeConnection) seedNodeConnection else defaultConnection } From 54b93e56a082258165c167d74da998f1e45fc31c Mon Sep 17 00:00:00 2001 From: Niels Andriesse Date: Fri, 14 May 2021 08:58:11 +1000 Subject: [PATCH 31/31] Update version number --- app/build.gradle | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/build.gradle b/app/build.gradle index 8b453260bb..06015ea874 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -158,8 +158,8 @@ dependencies { testImplementation 'org.robolectric:shadows-multidex:4.2' } -def canonicalVersionCode = 161 -def canonicalVersionName = "1.10.2" +def canonicalVersionCode = 162 +def canonicalVersionName = "1.10.3" def postFixSize = 10 def abiPostFix = ['armeabi-v7a' : 1,