This patch fixes the issue caused by #967, which broke the functionality of saving the retrieved data into a csv file.

* Fix ES publishing

* Remove hour() from elasticsearch.py

* fix for deprecation of v1.1 endpoints

* fix for cashtags

* typo

* fix(datetime): _formatDateTime tries %d-%m-%y

* fix(pandas): use new str-format Tweet.datetime data rep

* fix(pandas datetime): use ms

* fix(cashtags unwind): undo PRs field removals

* Revert "fix(cashtags unwind): undo PRs field removals"

This reverts commit dfa57c20186a969aa2bf010fbe198f5e0bbbbd01.

* fix(pandas): remove broken fields

* fix(cash): use provided field as suggested by pr review

* fix (cashtags): re enable cashtags in output

* fix(db): remove broken fields

* fix(datetime): Y-m-d and factored out

* fixes #947

* fix(get.py): json exception in User

* to-do: added to-do tasks

added to-do tasks for --profile-full feature

* chore(test): PEP8 formatting

* fix(profile): ported user profile to v2 API

fixed user profile feature which was broken since v1 endpoints were deprecated

* updated Readme

* fix: fixes #965 inconsistent timezones

* fix: handle tombstone tweets

tombstone tweets are those tweets which are flagged by Twitter for being inappropriate, misleading, graphic etc.

* fixes #976: saving tweets to csv

This patch fixes the issue caused by #967, which broke the functionality of saving the retrieved data into a csv file.

* feature: port Lookup to v2 endpoint

fixes #970, lookup is ported to v2 endpoint. this can now be used to lookup a certain profile.
Co-authored-by: SiegfriedWagner <mateus.chojnowski@gmail.com>
Co-authored-by: lmeyerov <leo@graphistry.com>

Twitter does not allow the tab character in tweets, so it is an ideal separator.  Separating with commas (i.e. without this option) results in ambiguity.

Co-authored-by: Friendly <Hacker>

Keeping the newlines resulted in invalid CSV.

Co-authored-by: Friendly <Hacker>

* fix for cashtags

* typo

* fix(datetime): _formatDateTime tries %d-%m-%y

* fix(pandas): use new str-format Tweet.datetime data rep

* fix(pandas datetime): use ms

* fix(cashtags unwind): undo PRs field removals

* Revert "fix(cashtags unwind): undo PRs field removals"

This reverts commit dfa57c20186a969aa2bf010fbe198f5e0bbbbd01.

* fix(pandas): remove broken fields

* fix(cash): use provided field as suggested by pr review

* fix(datetime): Y-m-d and factored out
Co-authored-by: SiegfriedWagner <mateus.chojnowski@gmail.com>

Co-authored-by: TimsXH <44843555+TimsXH@users.noreply.github.com>

See https://github.com/twintproject/twint/issues/913

* Add thumbnail attribute to tweet object

Fetch the video thumbnail of a tweet

* Add Thumbnail support for DB, CSV, JSON and Elasticsearch

* Update panda.py

Add missing datafields: video, photos, urls and thumbnail.

Update requirements - dataclasses missing

Following commit 5b152125

* favorite and following functions are working again using the user_id that fetched with tweet search function, also removed some of the user agents that returns dynamic site which is an undesired situation

* lookup url updated

* bio, location, url parsing ok

* returning join date empty since its not coming with the new interface

* inf function updated to new interface; name, username and private status fully ok, user_id will be parsed if there is a mention of the account itself on the first page

* stat function ok for tweets, following and followers counts. Favorite and media counts do not exist on the profile page

* verified function updated to new page

* updated User function, media count, likes, background image link are empty since they are not included in the new one
Co-authored-by: yunusemrecatalcam <yunus.catalcam@gmail.com>

favorite and following functions are working again using the user_id that fetched with tweet search function, also removed some of the user agents that returns dynamic site which is an undesired situation (#870)
Co-authored-by: yunusemrecatalcam <yunus.catalcam@gmail.com>

#831

With recent update of aiohttp_socks on July 16, 2020 - due to deprecation SocksConnector was renamed to ProxyConnector and SocksVer was renamed to ProxyType.

* Added language to tweetData and tweetFieldnames

* Added language to Tweet object as t.lang

* Added language to Tweet()

* added language field to update()

* updated to include language
Co-authored-by: JMorrow <jomorrcode@outlook.com>

fetching favorites and parsing in a new way since twitter returns a different page now. Output includes tweet text, tweet id, conversation id and date (#826)
Co-authored-by: yunus.catalcam <yunus.catalcam@vircongroup.com>

Co-authored-by: Lucas Magalhães <13581833+whoisroot@users.noreply.github.com>

* ability to specify tor control port and password from command line

* clarify documentation

* fixing not working example in README.md

* fixing error when username is not found

* revert README.md

* [FIX]: fix request error bug limiting requests

* [FEAT]: add dual poly. backoff + linear timer if request limit hit

Add a dual polynomial + linear timer to allow for optimal fine grain
searching of twitter timeout value. A new arg `min_wait_time` allows
users to to do one of three things; Firstly, if they know a value that
is optimal already then can input it directly allowing the timer to
stricty be a linear timer. Secondly, if Twitter suddenly changes their
timeout limit and the value no longer satifies the required amount,
the timer will decide the higher valued option between the polynomial
timer's and linear timer's next value so the application still
functions. Third, as the linear timers min_wait_time arg goes to zero
the timer becomes strictly a polynomial backoff timer that allows the
user versatility and less likely to explode in the case of a change in
Twitter request limit timeout.

* [FIX]: add args to config.py

* [REFACT]: minor changes to run.py

* [REFACT]: small change to arg case