This will stop instances of the following from occuring in the logs
on SMS migration:
W/SQLiteCompiledSql: Releasing statement in a finalizer. Please ensure
that you explicitly call close() on your cursor: INSERT INTO sms
(address, person, date_sent, date, protocol, read, status, type,
reply_path_present,
net.sqlcipher.database.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
at net.sqlcipher.database.SQLiteCompiledSql.<init>(SQLiteCompiledSql.java:62)
at net.sqlcipher.database.SQLiteProgram.<init>(SQLiteProgram.java:109)
at net.sqlcipher.database.SQLiteStatement.<init>(SQLiteStatement.java:39)
at net.sqlcipher.database.SQLiteDatabase.compileStatement(SQLiteDatabase.java:1647)
at org.thoughtcrime.securesms.database.SmsDatabase.createInsertStatement(SmsDatabase.java:767)
at org.thoughtcrime.securesms.database.SmsMigrator.migrateConversation(SmsMigrator.java:166)
at org.thoughtcrime.securesms.database.SmsMigrator.migrateDatabase(SmsMigrator.java:210)
at org.thoughtcrime.securesms.service.ApplicationMigrationService$ImportRunnable.run(ApplicationMigrationService.java:159)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
We aren't closing Statement objects before the finalizer on those
objects runs. When the GC runs, we'll get warnings like the above
which alert us to the fact that these objects are being automatically
closed for us in the finalizer, but that this is suboptimal behavior.
If we leave too many Statement (or Cursor) objects to be closed in
their finalizers, when the GC runs, it'll take longer than 10 seconds
to close them all and Android will kill the app. This 10 second limit
is hardcoded and we can only try to avoid it. A crash will look like:
java.util.concurrent.TimeoutException: net.sqlcipher.database.SQLiteCompiledSql.finalize() timed out after 10 seconds
at java.lang.Object.wait(Native Method)
at java.lang.Thread.parkFor$(Thread.java:1220)
at sun.misc.Unsafe.park(Unsafe.java:299)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:810)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:844)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1173)
at java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:196)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:257)
at net.sqlcipher.database.SQLiteDatabase.lock(SQLiteDatabase.java:553)
at net.sqlcipher.database.SQLiteCompiledSql.releaseSqlStatement(SQLiteCompiledSql.java:106)
at net.sqlcipher.database.SQLiteCompiledSql.finalize(SQLiteCompiledSql.java:152)
at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:202)
at java.lang.Daemons$FinalizerDaemon.run(Daemons.java:185)
at java.lang.Thread.run(Thread.java:818)
I was able to replicate the above crash consistently on a
Samsung Galaxy S7 edge when importing well over 100k SMS messages.
But as soon as I attached a debugger the crash did not persist. I
assume this is because of some VM-level interactions between the two
and did not investigate further after fixing it.
I do not have access to the stack trace for issue #7953 but this
could potentially resolve it. The crash is identical to that in #7477
but this patch is for SMS migration not restoring from a backup. I
was not able to replicate the crash on restoring a >100k message
backup.
It used to be that we let Android do the default behavior of
full-screening the EditText when in landscape, but honestly I
don't know who prefers that. So I've turned it off.
Fixes#8769
The new JobManager stuff created a table that had an
auto-incrementing ID, which was incorrectly being backed
up and restored, causing a crash. Now we skip it on both
import and export.
Some devices have a limit of 100 unique JobScheduler jobs.
Previously we allowed up to 1,000. Given that we just need
_some_ job running, I lowered the limit to 75 to give us
some head room.
- Eliminate the explicit spongycastle dependency. All access to
primitives is done through the JCE interfaces now, which allows
us to use a secure native-backed provider like conscrypt.
- Use conscrypt for our default security provider. This gives us
fast TLS 1.2 and 1.3 support on all devices, even before they
had platform support (like 4.4).
- Update minSdk to 18. Unfortunately the JCE interfaces for GCM
primitives are JDK 7+ (!) only, which became supported by Android
at 18.
We'll be updating minSdk to 19 in 4.37. This lets these users continue
to use the app, but they'll be warned with a persistent banner saying
that they can't receive updates.
If you navigated to the ConversationActivity again via some action (like
a shared contact invite) while searching, we don't get the toolbar close
event, and therefore the search nav would stay open. Now we just reset
it on newIntent() to be safe.
The way the highlight was done could get screwed up if you had multiple
whitespaces in a row. This particularly came up with messages with
multiple newlines.
There were situations where adding/removing members from a group
would update the group member list, but the short string (the little
text listing the first couple members of the group) wouldn't be updated
until you left the screen and came back.
1. Due to ShareActivity having noHistory=true, it will already be
ditched when you leave the activity.
2. We only need to truly finish() here if we've dropped the underlying
media.
Fixes#8591
Prevent users from trying to send videos that exceed the size limit.
Also, this commit properly populates height/width on media shared into
the app.
Fixes#8573
It didn't re-measure when pulling an item from the cache, screwing stuff
up after a phone rotation. Had a workaround for it for specific screens,
but this fixes the problem at the source.
Fixes#8583
There's odd corner cases where channels can be duplicated. This commit
adds some hard checks where we trim any dead channels, and unset any
notification channels from recipients whose notification channel isn't
present in the system settings.
`DEFAULT_CHANNEL_ID` is a String, but `channel` is a NotificationChannel. Equals will therefore always return `false`. I think my fix (using `getId()`) is what was intended.
TreeSets are annoying. contains() is calculated with the comparator,
which can lead to some weird bugs. Made sure the comparator didn't think
two items with the same date were identical.
Also fixed stableId generation to avoid any potential weirdness there.
Update our media send flow to allow users to send multiple images/videos
at once. This change includes:
- New in-app media picker flow.
- Ability to caption images and videos.
- Image editing tools are made more prominent in the flow.
- Some fixes to the image editing tools.
On some devices, pausing+resuming (or seeking) would restart playback
from the beginning, but show a progress bar further up. This seems to
have been caused by same race condition-y thing where we were seeking
multiple times in rapid succession. Now we'll only play once, and things
seem to be fine now.
Also removed usage of some deprecated methods.
Fixes#8432
The Android animators were getting out of sync when frames were dropped
(despite my best efforts), so now we just manually render each animation
frame as a function of time, so it never gets screwed up.
Fixes#8388
There are several (popular) phone models out there that have bugs in
their MediaPlayer implementation that cause them to be unable to play
voice notes. By moving to ExoPlayer, an application-level media player,
we should avoid most of these headaches and stardardize playback.
Fixes#7748
In the case where we add new fields to a Job's InputData, we want to
make sure that initialize() is called in the try block so that if it
fails, it simply fails the job (allowing the user to retry with the new
field) instead of crashing.
This is to guard against behavior WorkManager has where it may
re-enqueue a job that has already been completed (if, for instance, it
was preempted).
Fixes#8268
The aim of this is to help performance by breaking up the single massive
spritesheet into smaller ones. This will limit the amount of data that
needs to be kept in memory to render emoji.
(Hopefully) Fixes#8357
Because SMS sending is split over two jobs, there's no max retry limit
respected if we find out about the failure in SmsSentJob -- it's
requeued as a new job with a fresh attempt counter.
This commit carries a retry count between the two jobs. It also verifies
that we have service before attempting to send a message at all.
Relates to #8268
There are rare corner cases where a Job could be preempted by the
JobScheduler and then be rescheduled before the preempted job finished
running. This could create weird race conditions with bad consequences.
To fix this, we create a fun locking system that prevents two jobs with
the same UUID from running at the same time.
Includes display support for more genders, and more notably, skin tones.
These are not currently selectable in the UI, but they will be rendered
properly when other clients send them.
There's some ANRs, not many, that are likely caused by us riding the
5-second ANR timeout a little too closely. Giving us a little buffer to
see if that helps.
Some phones, notably the Pixel 3, had some problems with scaling after
taking photos. This fixes it by using the takePicture API instead of
pulling the bitmap from the TextureView.
Fixes#8292
Android P's new ringtone selector is a whole new activity that can cause
RecipientPreferenceActivity to go through the full onCreate() flow after
the ringtone selection. This could cause a race between setting the
preference and reading the preference from the notification channel.
Just threw them on a serial executor to guarantee ordering.
Due to an Android P bug, we basically need to stop calling
startService() in onResume()/onPause(). That means I had to turn
MessageRetrieval service into a singlton instead of a service. I also
moved the offending KeyCachingService calls into static methods that
didn't have to start the service.
If we already have two active processing GCM messages, there's no
benefit to a third. In fact, enqueuing additional ones will likely only
end up showing the foreground notification unnecessariliy.
Previously we were starting a background service, which isn't allowed
for targetSdk 26. This will do the same thing but at a time decided by
the system.
This gives us more control over when it happens, as well as lets us set
things like the debug level. Also let's us get rid of the synchronized
block we had in Application#onCreate().
This sound isn't supposed to play when you have notification sound off,
but apparently some Huawei phones will play it anyway. Until we can
figure out a better way to handle it, we're just removing it.
We don't store non-user-selected colors in the database. That means that
when we update the palette, we still have to hash based off of the legacy
palette when generating a color if we want to migrate to a
similar-looking color.
Unfortunately, because the new palette is smaller, some colors are
"overloaded", meaning that when we hash based off of the legacy palette,
some colors will be more/less common than others. To fix this, we simply
persist all current colors in the database, then switch our hashing list
to what we really want.
Occasionally a job may be run when the app is in a network-restricted
mode, like a form of doze. When this happens, jobs can timeout due to
lack of network access, causing a cascade of job delays. This is
particularly bad in the case of message retrieval.
To prevent this, if a job that normally requires network detects that no
network is available when running, then we start a foreground
notification.
Some devices will randomly throw RuntimeExceptions here due to hardware
issues. We were already doing broader catch statement in CameraView, so
I moved it here as well.
Unfortunately, while there does exist an XML property to disable the
indentation, it's bugged for category headings, so we have to do this
silly thing where we strip the padding in the adapter. Hopefully they'll
fix the bug and we can move to use the sanctioned property.
See: https://issuetracker.google.com/issues/111662669Fixes#8233
On older versions of Android, TextureView#getBitmap() needs to be called
on the main thread. On mid range phones, this is ~50ms. Normally that'd
be bad, but the UI isn't doing anything at that point anyway.
Fixes#8232
We were being inconsistent in how we were handling exiting multiselect,
and it wasn't behaving properly when you left by clicking the 'x'. Now
it's all handled centrally.
Fixes#8234
A new, fullscreen camera capture flow that easily allows you to capture
and edit a photo before sending it. Replaces the current half-screen
camera button.
Keep the single-use behavior, but allow the creation of multi-use memory
blobs that can be deleted when we're done with them. Will help out with
having URI's for temporary images during the camera capture flow.
This also fixes the situation where we block group-leave messages,
preventing blocked contacts from leaving groups.
Fixes#7970
Also, this forced us to upgrade libsignal-service, which fixes the
websocket timeout issues. Thanks to @dpapavas!
Fixes#6644
Forwarding of shared contacts currently just creates an empty draft.
You can't preview a shared contact before you send, which would make the
forwarding flow inconsistent across media types. So it's easier to just
hide it for now.
Fixes#8195
This could cause us to think push messages were actually sent with
a SIM. We also now prevent rendering SIM info on push messages in
the conversation view.
Fixes#8176
We were hitting the transaction limit size. This change scales down
shortcut icons to be at most 300x300, which comes out to ~360kb, which
should be safely under the limit of 1mb.
Fixes#8139
We were getting an IllegalArgumentException during channel creation on
some Samsung phones. Stack trace didn't give me much more than that, so
just adding in some additional safeguards that make sense based on
reading AOSP.
We manually play the ringtone when in-thread notifications are enabled,
but we weren't using the sound specified by the channel in the system
settings. This fixes that problem by reading the NotificationChannel
setting.
We were getting a TransactionTooLargeException when giving an
EditText a very large (1.5MB+) text block. This has been resolved
by switching to a RecyclerView to show the text line-by-line. As a
side-effect, this improves scroll performance on lower-end devices.
Also, I added a button to jump to the bottom of the log because I
really wanted one :)
Fixes#8124
Previously, we were making a new copy of the entire source string after
every scrubbed substitution. In the case of our new, larger log files,
this was very slow. It's been changed so we only ever create one new
copy.
In practice, on a Moto E (2014), scrubbing a 1.5MB log went from
>4000ms to ~100ms.
When using notification channels, us setting priority actually has
no effect. So instead of having a non-functional setting, we've
routed the notification priority setting to go to the system
notification channel settings page for our Messages channel.
Address linkification on Android is pretty busted, and the docs
acknowledge it (see Linkify#MAP_ADDRESSES). Safest thing to do
at the moment is remove it. Looks like we may be able to get
better address linkification on API >= 28, but adding it will
be more involved.
Fixes#7730
Previously, we'd only show the attachment button when the user had
yet to enter any text. To add an attachment after text was entered,
you'd have to go to the three-dot menu. Now we just show a little
attach button in the text area.
I also took the opportunity to clean up other button paddings and
stuff in the compose area so things look better and react to text
sizes more predictably.
Added a new logger that persists logs for a longer duration to the
user's cache directory. Logs are encrypted. The new logs are sent
in addition to the user's logcat output.
For short messages in a cluster, the contact photo was sometimes taller
than the actual bubble, leading to extra weird space. So instead we use
a container to hold the width of the cell, and set the avatar to be GONE
instead of INVISIBLE.
The previous way we were getting the next/previous record didn't take into
consideration that some records aren't in the cursor -- some are in the
fastRecords map. We now use the proper position to get the next/previous
message.
Apparently onAnimationEnd is not a reliable event on some Android
versions, so I've moved to instead using a simple postDelayed() that is
the same length as the animation.
The media size traditionally determines the bubble size, but the author
could make it wider, which would lead to rendering issues. In the case
of media attachments (images, videos, and shared contacts), we restrict
the width of the author name. When there's a number+profile name combo,
we split the space 50/50 between the two.
In particular, there were many issues with drawing corners.
Unfortunately, there's no pretty way to get masking working on every
Android version, so we have to switch back to using custom backgrounds
and then using multiple masking methods depending on Android version.
Also, I had to remove attr references in drawables. They crash on 4.x.
Unfortunately, there's apps out there that trigger contact changes
very frequently. Because we listen to the system for contact
changes to tell us when to sync, that could result in us sending
an abundance of contact syncs to linked desktop instances.
This throttles contact sync requests using the following methodology:
- By default, throttle contact syncs to 6 hrs while the app is
backgrounded.
- If a sync is throttled in the background, we set a dirty flag and
will execute the sync the next time the app is foregrounded.
- Syncs explicitly requested by desktop are never throttled.
The directory we were previously saving backups to on the external SD
card is actually deleted upon app uninstall and/or clearing the app's
data. There's also no reliable way to write to the root of an external
SD card (that isn't comically inconvenient), so for now it's safer if we
just move back to getting the regular 'ol standard external storage
directory (which is likely internal storage, despite its name).
Fixes#7845
For self-sends, we were never marking attachments as uploaded. I made is
so that happens now, but to prevent it for showing for already-sent
messages, we also don't show controls for self-send conversations.
This particularly helps with the bug where people who were newly added
to a group wouldn't receive an expiration timer until the first message
was sent.
Previously, contact shares would be displayed as "Media Message". Now
it'll show the same as it does in a notification, namely
"{contact-emoji} {contact-name}".
Previously, we were running this job in PushSendJob#onCanceled().
However, with the new retry logic, this won't happen for 24 hours.
Instead, we now schedule the job in PushSendJob#onRetry().
Previously, we retried based on a count. Now we've added the ability to
keep retrying for a specified time, using exponential backoff to
throttle attempts.
We never properly registered the ExpirationListener, meaning we were
relying on the wait-notify loop of ExpirationManager to delete things.
This normally works, but fails when your phone goes to sleep. So I
properly registered the receiver, and then added a failsafe to re-run
the ExpirationManager if we're about to render an expired message.
Fixes#7906
We have to make some changes, and it's gotten to the point where
maintaining it as a separate library is more hassle than it's worth,
especially with Google releasing WorkManager as the preferred job
scheduling library.
Previously, because apostrophes were 'banned' characters, searching for
them wouldn't work. That meant you couldn't find words like "I'm". Now
we just replace the apostrophe with a space and things "just work"
because of the nature of SQLite tokenization and prefix queries.
Previously, we didn't support highlighting search results that had
tokens in the middle of the matches, which is a possibility with FTS.
Now we do more robust highlighting, as well as highlight matches in
phone numbers.
Previously, we made each full-text search query a single prefix query.
That means that the query "do c" would turn into "do c*". That means it
would match "do cat" but not "dog cat".
Now, we make each token a prefix query. So "do c" would turn into
"do* c*". That means it would match both "do cat" and "dog cat".
Previously, if a message disappeared while looking at it in the search
results, it'd still stick around. Now they'll disappear from the results
in real-time.
1) There was an issue where we wouldn't auto-download group syncs.
2) There was another issue where we didn't show the download controls
for messages you sent yourself.
Fixed#7920
There was an issue where we were backing up group receipts and attachments
that were for expiring messages (which are already excluded from the backup).
This commit excludes these items from the backup, and for backups made
before this change, this commit also deletes these invalid entries at
the end of the restore process.
We also do a little database migration to cleanup any bad state that may
have been imported in the past.
Turns out that there's some weird quasi-state when you come out of
airplane mode, that if you do an InetAdress lookup, it returns some
weird IPv6-looking garbage address. Going to retry in that scenario
instead of assuming an outage.
Previously, if an operation failed, we set the result to "null".
However, this was a mistake. Setting the result at all assumes success.
Instead, we need to set an exception so the ListenableFuture knows that
the operation failed.
A variety of improvements to the image editor, such as:
- New, fullscreen styling
- Smoother lines
- Better text and sticker handling
- Improved color picker with a history pallette
- New highlighter tool
Previously, SQLCipher's memory usage would grow indefinitely, up until
it hit the end of the cursor. We've now switched to a release where the
memory used by the cursor can be bounded.
If a user is upgrading to use the SQLCipher database (which happened
back in 4.16, so this only applies to relatively dormant users who are
just getting back into the app) and received a new "user X joind signal"
message, then it could screw up the migration. So we're just dropping
these notifications that happen in this narrow window.
The sticky header cache was keeping views across rotations, causing them
to render incorrectly afterwards. I added a method to invalidate the
header layouts after rotation.
Fixes#7890.
First, FTS index contents do not need to be exported. They will be recreated naturally.
Second, we can't export the secret FTS tables, or SQLite will think it's corrupted.
Fixed an issue where jumbomoji were not properly being rendered
when using system emoji. Also fixed an issue where the text
content wasn't properly being recalculated when the view is
resized.
Fixes#7875
1. Switch to using default text rendering if there's no emoji present in
the string.
2. Reduce redudant redraws by skipping of setText() calls for identical
strings.
Together, these two changes should reduce the vast majority of
flickering we see with EmojiTextView ellipsizing.
I think I was initially lured into searching by rank because it gives
the illusion of providing the "best match". However, in practice, FTS
never gives back "bad" matches with low ranks -- all of the results it
returns will contain your query in some form (most commonly a direct
substring, but they do take some liberties if you have multiple tokens
in your queries). Given that, in general, more recent search results are
in fact more relevant, we can sort by date exclusively and get a better
ordering overall.
If a search result snippet spans two lines, we only show the first line.
For the purpose of display, we first remove all newlines in order to
make the full snippet visible.
Previously, we'd always use the recipient of the message record, which
was incorrect for messages sent to groups. Now we always use the
recipient for the matching thread record.
Fixes#7823
Glide will use the dimensions of the target ImageView as the dimensions
for the image it's loading. This caused problems in the case of
ThumbnailView, as we were constantly changing the ImageView dimensions,
meaning Glide may not have the most recent values (it may be called in
between measure calls, for instance).
To solve this, we now will always override the default image dimensions
when we load an image. If no dimensions are present, we will default to
the layout_width and layout_height of the ThumbnailView.
Fixes#7810
Notifications for replies will no longer display as a "Media Message" if
they do not contain media. Instead, they will just contain the reply
text.
Fixes#7798
