DE102025131074A1 - Communication system with simultaneous translation - Google Patents

Abstract

A communication system for acoustically based communication for exchanging speech content between at least two participants, each equipped with a communication terminal (10a, 10b, 10c, 10d), via an intermediate data server (12), wherein the data server comprises a translation device (22) for synchronously providing a translation of a speech content from the language of the speech content generator into a receiver language of the at least one speech content receiver, as well as a voice generator (24) for outputting the translated speech content, wherein the communication terminal (10a, 10b, 10c, 10d) of the at least one speech content receiver acoustically transmits the translated speech content to the speech content receiver, is characterized in that the communication is a teleconference with at least three participants and the voice generator (24) is designed and configured to output the translated speech content to the speech content receiver with a virtually generated voice pitch, which is based on properties of the voice pitch of the Speech content generator and differs from the virtually generated voice tones of the other participants.
This assigns each participant in the teleconference a virtually generated voice pitch for outputting the translated language content, which makes it as easy as possible to distinguish between the participants.

Description

The invention relates to a communication system for acoustically based communication for exchanging voice content between at least two participants, each equipped with a communication terminal, via an intermediate data server. The data server comprises a translation device for synchronously providing a translation of a voice content from the language of the voice content generator into a receiver language of the at least one voice content receiver, as well as a voice generator for outputting the translated voice content. The communication terminal of the at least one voice content receiver acoustically transmits the translated voice content to the voice content receiver. The invention further relates to a motor vehicle comprising an installed communication terminal.

In this context, a "communications terminal" is understood to mean a mobile or (preferably in a vehicle) built-in communications device with a microphone, loudspeaker, and possibly a camera. This includes, in particular, computers, mobile devices such as smartphones or tablets, data glasses, or infotainment systems in vehicles. These must simply be capable of establishing a wired or wireless data connection to the data server.

From the DE 10 2012 019 994 A1 Such a communication system is known which can be used in a motor vehicle to allow two conversation partners to communicate acoustically with each other using different languages, whereby both participants receive the speech content of the other participant translated into their language.

If not just two but several participants conduct an audio or video conference with each other, for example using MS Teams, Google Meet, Zoom or Skype, it is often difficult for a recipient of voice content to determine which participant is currently speaking, i.e. transmitting voice content.

The object of the invention is to provide a communication system for an audio or video conference with at least three participants, in which the assignment of a received speech content to a participant is made easier for the speech content receivers.

The invention is based on the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims.

The object is achieved according to claim 1 in that the communication is a teleconference with at least three participants and the voice generator is designed and configured to output the translated speech content to the speech content receiver with a virtually generated voice pitch that is generated on the basis of properties of the voice pitch of the speech content generator and differs from the virtually generated voice pitches of the other participants.

In this context, teleconference is understood to mean a video or audio conference with at least three participants.

The invention assigns each participant in the teleconference a virtually generated voice pitch for outputting the translated speech content, which makes it as easy as possible to distinguish between the participants. The more participants participating in the teleconference, the easier it is for each participant to assign the selected voice pitches to the individual participants after a short period of familiarization. Preferably, the voice generator selects virtually generated voice pitches for different participants with similar characteristics of their real voice pitches that are easily distinguishable in terms of sound. The translation device is configured to perform a plurality of simultaneous translations for a plurality of speech contents simultaneously.

According to an advantageous development, the at least one communication terminal comprises a contact data memory for storing contact data of persons, wherein the contact data comprises information about a preferred language of the person and/or the gender of the person and/or an indication of the person's age. When a teleconference is set up with a person, this information can be transmitted from the contact data memory to the data server. The translation device is configured to output a virtually generated voice pitch for this person on the basis of this information when outputting translated language content. As soon as the contact data of the teleconference participants are available in the data server, the data server automatically knows the preferred language of all participants, even if the individual participants do not know the preferred language of the other participants.

According to an advantageous further development, each participant can also individually change the virtual voice pitch of the other participants via their communication device, for example, selecting a virtual voice pitch from a set of voice pitches. This further development is useful if a participant is to change the virtual voice pitch at different times (at different different days) with individual recurring participants and then it improves the acoustic recognition of the participants if the communication system uses the same virtual voice tones for the same participants each time.

According to an advantageous embodiment of the invention, the voice generator comprises a gender detection device configured to determine the gender of the speech content producer from the pitch of their speech content. The voice generator outputs the translated speech content to the at least one speech content receiver with a virtual pitch assigned to the gender. This improves the distinguishability of the participants. If a video conference is involved, image processing of the video image (preferably also AI-based) can also perform or supplement the participant's gender determination.

According to an advantageous development of the invention, the voice generator comprises an age detection device configured to classify the age of the voice content generator, wherein the voice generator outputs the translated voice content to the at least one voice content receiver with a virtual voice pitch assigned to the age. A distinction can be made, for example, in three or four age categories. Since the voices of older people sound different from those of younger people, different participants can be easily distinguished by selecting a virtual voice pitch corresponding to the approximate age of the actual participant. The age detection device can perform the age classification by preferably analyzing the voice pitch of a participant using AI. If a video conference is involved, image processing of the video image (preferably also AI-based) can also perform or supplement the age classification.

According to an advantageous development of the invention, when a subscriber establishes a communication, the communication terminal can transmit the subscriber's age information stored therein to the data server, whereby the voice generator outputs the translated speech content of the speech content generator to the at least one speech content receiver with a virtual pitch associated with the age information. Instead of using an age detection device, it is more accurate and less error-prone if age information is transmitted directly from the communication terminal to the voice generator.

For example, the age information can be contained in a participant's contact record, for example, in an extension of the standard vCard (vcf) format, which is transmitted to the voice generator. The data from such a contact record can also be forwarded by the data server to all other participants, allowing their communication device to select a virtual voice range for a known and stored contact record. This selection of a specific virtual voice range can be transmitted back to the data server, ensuring that the same virtual voice range is always assigned to the known participant in every future teleconference.

According to an advantageous development of the invention, when a subscriber establishes a communication, the communication terminal transmits gender information of the subscriber stored therein to the data server, wherein the voice generator outputs the translated speech content of the speech content generator with a virtual voice pitch associated with the gender information to the at least one speech content receiver.

For example, gender information can also be included in a participant's contact record, for example, in an extension of the standard vCard format, which is transmitted to the voice generator. The data from such a contact record can also be forwarded by the data server to all other participants, allowing their communication devices to select a virtual voice pitch for a known and stored contact record.

According to an advantageous development of the invention, the voice generator comprises an AI-based eigenvoice generator, which can be trained based on a participant's training speech content to output the translated speech content with a virtually generated voice pitch that imitates the voice pitch of the speech content generator. Thus, according to a development, before the start of a teleconference, a voice file previously created by the participant according to predetermined criteria and stored in the communications terminal could be transmitted to the eigenvoice generator to obtain the participant's imitated virtual voice pitch.

According to an advantageous development of the invention, when a subscriber sets up a communication, the communication terminal can transmit a receiver language stored therein to the data server, which then translates all translated language content from the other subscribers into the receiver language. This recipient language can be saved as part of a vCard and transmitted to the data server when a teleconference is established, so that the translation facility automatically selects the desired language for the participant and the language content of the other participants is translated into this preferred language.

According to an advantageous development of the invention, the communication terminal includes a switching function configured to individually switch the translation function for each other participant in order to transmit the speech content of the other participants to the user of the communication terminal either in the original or translated, depending on the switching position. Thus, a participant who is fluent in another language can preferentially capture the speech content in the original language of the speech content generator to avoid translation inaccuracies.

According to a second aspect of the invention, it comprises a motor vehicle in which a communications terminal according to one of the preceding aspects is installed. This makes it possible for a passenger of the motor vehicle to conduct video or audio chats with several people or participate in teleconferences via an installed infotainment system and to clearly distinguish the other participants acoustically without distracting their visual attention from the traffic.

Further advantages, features, and details will become apparent from the following description, which describes at least one embodiment in detail with reference to the drawings. Identical, similar, and/or functionally equivalent parts are provided with the same reference numerals.

• In der Figur sind mehrere Kommunikationsendgeräte 10a, 10b, 10c, 10d dargestellt, die mit einem Datenserver 12 in Kommunikationsverbindung stehen. Die Kommunikationsendgeräte 10a, 10b können mobile Endgeräte wie Smartphones oder Tablets sein, die über drahtlose Verbindungen 14 mit dem Datenserver 12 kommunizieren, beispielsweise über ein Mobilfunknetz. Das Kommunikationsendgerät 10c ist ein stationärer Computer, der über eine Datenleitung 16 mit dem Datenserver 12 verbunden ist. Das Kommunikationsendgerät 10d ist ein in einem Kraftfahrzeug 18 verbautes Endgerät mit einem Fahrzeugdisplay 20, der über eine drahtlose Verbindung 14 mit dem Datenserver 12 kommuniziert. Die Kommunikationsendgeräte 10a, 10b, 10c, 10d umfassen jeweils mindestens eine Applikation (App) zur Durchführung einer Telekonferenz, beispielsweise Skype oder eine entsprechende proprietäre App eines OEMs (insb. eines Kraftfahrzeugherstellers). Die Kommunikationsendgeräte 10a, 10b, 10c, 10d weisen vorzugsweise jeweils eine Kamera auf, um Videokonferenzen durchführen zu können.

The figure shows a schematic representation of the communication system according to the invention.

• The figure shows several communication terminals 10a, 10b, 10c, 10d that are in communication connection with a data server 12. The communication terminals 10a, 10b can be mobile devices such as smartphones or tablets that communicate with the data server 12 via wireless connections 14, for example, via a cellular network. The communication terminal 10c is a stationary computer connected to the data server 12 via a data line 16. The communication terminal 10d is a terminal installed in a motor vehicle 18 with a vehicle display 20 that communicates with the data server 12 via a wireless connection 14. The communication terminals 10a, 10b, 10c, 10d each comprise at least one application (app) for conducting a teleconference, for example, Skype or a corresponding proprietary app from an OEM (in particular, a motor vehicle manufacturer). The communication terminals 10a, 10b, 10c, 10d preferably each have a camera in order to be able to conduct video conferences.

The data server 12 can be assigned to a teleconferencing platform such as MS Teams, Google Meet, Zoom, or Skype, or even to an OEM, and comprises at least one translation device 22 based on a generative AI model, which is configured to perform a plurality of simultaneous translations from one language from an available set of translation languages into a second language. The translation device 22 can be based, in particular, on the speech-to-text (STT) and text-to-speech (TTS) principle. This comprises a voice generator 24, which comprises a plurality of voice pitches generated by training with different voices for generating speech outputs in order to output these speech outputs with one of the stored voice pitches. The stored voice pitches each comprise several sonically distinguishable voice pitches for the two genders (male, female) and different age categories. Possible age categories include 0-10 years (children), 11-20 years (teenagers), 21-40 years (young adults), 41-60 years (middle-aged adults), 61-80 years (old adults), 80 and over (very old adults). More or fewer age categories may be provided. Since, depending on the scope of the conference, in teleconferences with multiple participants, several participants may fall into the same gender and age category, several voice ranges are stored in the voice generator 24 for each age and gender category.

The voice generator 24 may comprise a gender detection device 26 configured as a trained AI model to determine the gender of the speaker based on spoken text of a participant.

The voice generator 24 may comprise an age detection device 28 which is configured as a trained AI model to determine the approximate age of the speaker on the basis of spoken text of a participant and to classify it into one of the above-mentioned categories.

The voice generator 24 may alternatively or additionally comprise an eigenvoice generator 30 comprising a Kl model that is trained on the basis of spoken text of a participant and generates therefrom a virtually generated Voice pitch is generated and stored, which imitates the voice pitch of the participant.

The translation device 22 comprises a speech recognition device 32, which is configured as a trained AI model to determine the language of a speaker on the basis of spoken text of a participant, preferably based on a speech-to-text functionality.

The communication terminals 10a, 10b, 10c, 10d contain at least one app for storing contact data, in particular names, addresses, telephone numbers, and email addresses, preferably in the form of a digital business card 34, often in a so-called vCard file format (short for Versit card, after the publisher of the file format Versit), also referred to as a VCF file format, or in the form of a digital calendar, for example, in the iCalender file format. In a further development of the invention, such a digital business card 34 can include additional fields, namely for the gender, age (or year or decade of birth), and/or a preferred language of the contact person.

If a video or audio teleconference is conducted with multiple participants using digital business cards 34, according to a first embodiment, the teleconference organizer transmits the business card data regarding the gender, age, and preferred language of the people participating in the teleconference to the data server 12, where the translation device 22 causes the voice generator 24 to translate the voice content of all other participants for each participant into the preferred language of each individual participant according to the business card data. Furthermore, the self-voice generator 30 is controlled to select a voice pitch for all translations of this participant's voice content that corresponds to the gender and age according to the business card data.

For example, for a teleconference with four participants organized by participant 1, the data server 12 receives the information shown in Table 1 from the business card data of the other three participants. Table 1 Participant Gender Old Preferred Language 1 M 30 DE 2 W 50 EN 3 M 40 IT 4 W 60 EN 5 M 25 DE

Each participant receives the other participants' original language content, provided their preferred language is the same as the language of the content. If participant 1 produces German language content, it is translated into English to participant 2 and delivered in a voice pitch corresponding to a young male adult. The same language content is translated into Spanish to participant 3 and delivered in the same voice pitch. The same language content is translated into English to participant 4 and delivered in the same voice pitch. The same language content is delivered untranslated to participant 5.

If participant 2 creates an English language content, it is translated into German to participant 1 using a middle-aged female voice.

If participant 5 creates a German language content, this is translated into English to participant 2, for example, and output in a voice that is significantly different from the voice of participant 1, who falls into the same gender and age category as participant 5.

Each participant also has the option of switching the translation on or off for each other participant, preferably using a display switch 36 assigned to each other participant, i.e. receiving the language content in the original instead of the translation of the language content of this participant.

Each communication terminal 10a-10d further has a language selector 38 for setting the preferred language of the user of this communication terminal 10a-10d. This setting takes precedence over the preferred language specified by the teleconference organizer, so that the translated language content of the other participants is transmitted in this language set by the user of the communication terminal 10a-10d and not in any other language.

According to a second embodiment, the teleconference organizer transmits no data to the other participants other than the organizer's preferred language. However, according to a further development, a training text from the participant can be transmitted to the data server 12. so that the eigenvoice generator 30 can generate a virtual voice pitch of the participant by means of AI training, which is used for all translations to the other participants

According to a third embodiment, as in the second embodiment, the teleconference organizer transmits no data to the other participants other than the organizer's preferred language. The voice generator 24 assigned to the data server 12 determines the gender of a participant from a speech content via the gender detection device 26 and the approximate age of the speech content producer via the age detection device 28. It generates a virtual voice pitch for further speech content from the speech content producer, which is used for translating the further speech content of this participant or speech content producer.

Although the invention has been illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived therefrom by those skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that embodiments mentioned by way of example really only represent examples that are not to be understood in any way as limiting the scope of protection, possible applications, or configuration of the invention. Rather, the foregoing description and the description of the figures enable those skilled in the art to specifically implement the exemplary embodiments. The skilled person, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

List of reference symbols

10a - d

Communication devices

12

Data server

14

wireless connections

16

data line

18

motor vehicle

20

Vehicle display

22

Translation facility

24

Voice generator

26

Gender registration facility

28

Age registration facility

30

Eigenvoice generator

32

Speech recognition device

34

digital business card

36

Display switch

38

Language settings

QUOTES CONTAINED IN THE DESCRIPTION

This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 10 2012 019 994 A1 [0003]

Claims (10)

Communication system for acoustically based communication for exchanging speech content between at least two participants, each equipped with a communication terminal (10a, 10b, 10c, 10d), via an intermediate data server (12), wherein the data server comprises a translation device (22) for synchronously providing a translation of a speech content from the language of the speech content generator into a receiver language of the at least one speech content receiver, as well as a voice generator (24) for outputting the translated speech content, wherein the communication terminal (10a, 10b, 10c, 10d) of the at least one speech content receiver acoustically transmits the translated speech content to the speech content receiver, characterized in that the communication is a teleconference with at least three participants and the voice generator (24) is designed and configured to output the translated speech content to the speech content receiver with a virtually generated voice pitch, which is based on properties of the voice pitch of the Speech content generator and differs from the virtually generated voice tones of the other participants. Communication system according to Claim 1 , characterized in that at least one communication terminal (10a, 10b, 10c, 10d) comprises a contact data memory (34) for storing contact data of persons, wherein the contact data comprises information about a preferred language of the person and/or the gender of the person and/or an age of the person and when starting a teleconference with a person from the contact data memory (34) this information can be transmitted to the data server (12) and the translation device (22) is set up to output a virtually generated voice pitch for this person when outputting translated language content on the basis of this information. Communication system according to one of the preceding claims, characterized in that the voice generator (24) comprises an age detection device (28) which is set up to classify the age of the speech content generator, wherein the voice generator (24) outputs the translated speech content with a virtual voice pitch assigned to the age to the at least one speech content receiver. Communication system according to one of the preceding claims, characterized in that the at least one communication terminal (10a, 10b, 10c, 10d) transmits, when a communication is established by a subscriber, age information of the subscriber stored therein to the data server (12), wherein the voice generator (24) outputs the translated speech content of the speech content generator with a virtual voice pitch associated with the age information to the at least one speech content receiver. Communication system according to one of the preceding claims, characterized in that the at least one communication terminal (10a, 10b, 10c, 10d) transmits gender information of the subscriber stored therein to the data server (12) when a communication is established by a subscriber, wherein the voice generator (24) outputs the translated speech content of the speech content generator with a virtual voice pitch assigned to the gender information to the at least one speech content receiver. Communication system according to one of the preceding claims, characterized in that the voice generator (24) comprises a KI-based eigenvoice generator which can be trained on the basis of training speech contents of the speech content generator to output the translated speech content with a virtually generated voice pitch which imitates the voice pitch of the speech content generator. Communication system according to one of the preceding claims, characterized in that the data server (12) comprises a speech recognition device (32) which is set up to determine the language of the speech content on the basis of speech content of the speech content generator and to determine this as the recipient language of the speech content generator. Communication system according to one of the preceding claims, characterized in that the at least one communication terminal (10a, 10b, 10c, 10d) transmits, when a teleconference is set up by a participant, a receiver language stored therein to the data server (12), which translates all translated language contents transmitted to this participant from the other participants into the receiver language. Communication system according to one of the preceding claims, characterized in that the at least one communication terminal (10a, 10b, 10c, 10d) comprises a switching function (36) which is set up to switch the translation function individually for each other subscriber in order to transmit speech contents of the other subscribers, depending on the switching position, either in the original or translated to the user of the communication terminal (10a, 10b, 10c, 10d). Motor vehicle comprising a built-in communication terminal (10d) according to one of the preceding claims.