WO2011094927A1

WO2011094927A1 - Method and apparatus for automated mashup tool

Info

Publication number: WO2011094927A1
Application number: PCT/CN2010/070471
Authority: WO
Inventors: Jinghai Rao; Huajun Chen
Original assignee: Nokia Inc
Current assignee: Nokia Inc
Priority date: 2010-02-02
Filing date: 2010-02-02
Publication date: 2011-08-11
Anticipated expiration: 2012-08-02

Abstract

Techniques for an automated mashup tool include determining a local source, on user equipment, for at least one of functionality or data. A parameter of the local source is caused to be presented on the user equipment. Furthermore, determination is caused of a second parameter of a different source to be used with the parameter of the local source in an application that serves as a new source for a network to which the user equipment can be connected.

Description

METHOD AND APPARATUS FOR

AUTOMATED MASHUP TOOL

BACKGROUND

[0001] A mashup is a web page or other network application that combines data or functionality from two or more external sources to create a new service. The term mashup implies fast and easy integration, frequently using an open application programming interface (API) for each of the external applications and data sources to produce results that were not the original reason for producing the raw source data. An example of a mashup is the use of cartographic data to add location information to real estate data, thereby creating a new and distinct web service (with or without its own API) that was not originally provided by either source.

SOME EXAMPLE EMBODIMENTS

[0002] Applicants have recognized a need for a mashup tool that includes one or more sources on a user's own device (called local sources herein) or otherwise automatically assists a user in forming a mashup of several sources. As used herein, a source refers to an application that provides functionality or a structure that provides data or some combination. Therefore, there is a need for an improved mashup service with one or more automated assists.

[0003] According to one embodiment, a method comprises determining a local source of functionality or data on user equipment. The method also comprises causing a parameter of the local source to be presented on the user equipment. The method further comprises causing determination of a second parameter of a different source to be used with the parameter of the local source in an application that serves as a new source for a network to which the user equipment can be connected.

[0004] According to another embodiment, an apparatus comprising at least one processor, and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause, at least in part, the apparatus to determine a local source of functionality or data on user equipment. The apparatus is also caused to cause a parameter of the local source to be presented on the user equipment. The apparatus is further caused to cause determination of a second parameter of a different source to be used with the parameter of the local source in an application that serves as a new source for a network to which the user equipment can be connected.

[0005] According to another embodiment, a computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause, at least in part, an apparatus to determine a local source of functionality or data on the user equipment. The apparatus is also caused to cause a parameter of the local source to be presented on the user equipment. The apparatus is further caused to cause determination of a second parameter of a different source to be used with the parameter of the local source in an application that serves as a new source for a network to which the user equipment can be connected.

[0006] According to another embodiment, an apparatus comprises means for determining a local source of functionality or data on user equipment. The apparatus also comprises means for causing a parameter of the local source to be presented on the user equipment. The apparatus further comprises means for causing determination of a second parameter of a different source to be used with the parameter of the local source in an application that serves as a new source for a network to which the user equipment can be connected.

[0007] According to another embodiment, a method comprises determining a first source of functionality or data on a network. The method also comprises causing a parameter of the first source to be presented on user equipment. The method further comprises causing automatic determination of a different source based on semantic analysis of the first source and the second source, wherein a second parameter of the second source is capable of being used with the parameter of the first source in an application that serves as a new source on the network.

[0008] According to various other embodiments, a computer-readable storage medium or an apparatus is configured to perform one or more steps of the preceding method. [0009] According to various other embodiments, a method comprises facilitating access, including granting access rights, to an interface to allow access to a service via a network, the service comprising one or more steps of the above methods.

[0010] Still other aspects, features, and advantages of the invention are readily apparent from the following detailed description, simply by illustrating a number of particular embodiments and implementations, including the best mode contemplated for carrying out the invention. The invention is also capable of other and different embodiments, and its several details can be modified in various obvious respects, all without departing from the spirit and scope of the invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings:

[0012] FIG. 1 is a diagram of a system for an automated mashup tool, according to one embodiment;

[0013] FIG. 2 is a diagram of the components of an automated mashup tool, according to one embodiment;

[0014] FIG. 3A is a diagram of an application programming interface (API) data structure, according to one embodiment;

[0015] FIG. 3B is a diagram of a source metadata data structure, according to one embodiment;

[0016] FIG. 3C is a diagram of a mashup histories data structure, according to one embodiment;

[0017] FIG. 4A is a flowchart of a process for a mashup service to support an automated mashup tool, according to one embodiment; [0018] FIG. 4B is a flowchart of a process for a path recommender module to support an automated mashup tool, according to one embodiment;

[0019] FIGs. 5A-5F are diagrams of user interfaces utilized in the processes of FIG. 4A, according to various embodiments;

[0020] FIG. 6 is a diagram of hardware that can be used to implement an embodiment of the invention;

[0021] FIG. 7 is a diagram of a chip set that can be used to implement an embodiment of the invention; and

[0022] FIG. 8 is a diagram of a mobile terminal (e.g., handset) that can be used to implement an embodiment of the invention.

DESCRIPTION OF SOME EMBODIMENTS

[0023] Examples of a method, apparatus, and computer program for an automated mashup tool are disclosed. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It is apparent, however, to one skilled in the art that the embodiments of the invention may be practiced without these specific details or with an equivalent arrangement. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments of the invention.

[0024] As used herein, the term a source refers to an application that provides functionality or a structure that provides data or some combination; and each source includes one or more parameters for inputting data to the source or outputting data from the source and one or more operations that provide source functionality. As used herein, the term mashup tool refers to a module for forming a new source based on one or more extant sources; and the term local mashup tool refers to a mashup tool that includes one or more extant sources on a device where at least a portion of the local mashup tool is implemented. [0025] Although various embodiments are described with respect to a mashup server on a remote device connected to a local device through a network, it is contemplated that the approach described herein may be used with other components, such as a mashup application functioning entirely on a local device or multiple modules of the mashup server each executing on a different device connected to the network.

[0026] FIG. 1 is a diagram of a system 100 for an automated mashup tool, according to one embodiment. As indicated above, prior mashup tools allow a user to specify one or more network sources to incorporate in a new source, but not include many local sources. Furthermore, such mashup tools require that the user is knowledgeable about details of application programming interfaces (APIs) of the network sources, so that the proper APIs, parameters, and parameter usage can be indicated to the mashup tool. Moreover, the prior mashup tools provide little automatic support based on prior experience garnered during previous mashup operations performed by the user or others. Other prior mashup tools do not allow sources to be used without a published API, thus making unavailable for mashup many web pages otherwise available to the public.

[0027] To address this problem, the system 100 of FIG. 1 introduces an automated mashup tool with the capability to include local sources, and, in some embodiments, provide automated support in identifying sources and parameters to include in a new source to be built with the mashup tool. The system 100 includes a mashup service 120 tool that identifies applications and data on a user's equipment 101; provides semantic processing of text associated with various sources; and maintains histories of prior mashup operations by the user and others.

[0028] As shown in FIG. 1, the system 100 comprises a user equipment (UE) 101 having connectivity to mashup service 120 via a communication network 105. By way of example, the communication network 105 of system 100 includes one or more networks such as a data network (not shown), a wireless network (not shown), a telephony network (not shown), or any combination thereof. It is contemplated that the data network may be any local area network (LAN), metropolitan area network (MAN), wide area network (WAN), a public data network (e.g., the Internet), short range wireless network, or any other suitable packet- switched network, such as a commercially owned, proprietary packet- switched network, e.g., a proprietary cable or fiber-optic network, and the like, or any combination thereof. In addition, the wireless network may be, for example, a cellular network and may employ various technologies including enhanced data rates for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., as well as any other suitable wireless medium, e.g., worldwide interoperability for microwave access (WiMAX), Long Term Evolution (LTE) networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (WiFi), wireless LAN (WLAN), Bluetooth™, Internet Protocol (IP) data casting, satellite, mobile ad-hoc network (MANET), and the like, or any combination thereof.

[0029] The HE 101 is any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, cell phone, station, unit, device, multimedia computer, multimedia tablet, Internet node, communicator, desktop computer, laptop computer, Personal Digital Assistants (PDAs), audio/video player, digital camera/camcorder, positioning device, television receiver, radio broadcast receiver, electronic book device, game device, or any combination thereof. It is also contemplated that the HE 101 can support any type of interface to the user (such as "wearable" circuitry, etc.).

[0030] By way of example, the HE 101, mashup service 120 and other network services, such as network service 110a, other network services indicated by ellipsis, and network service 11 On collectively referenced hereinafter as network services 110, communicate with each other and other components of the communication network 105 using well known, new or still developing protocols. In this context, a protocol includes a set of rules defining how the network nodes within the communication network 105 interact with each other based on information sent over the communication links. The protocols are effective at different layers of operation within each node, from generating and receiving physical signals of various types, to selecting a link for transferring those signals, to the format of information indicated by those signals, to identifying which software application executing on a computer system sends or receives the information. The conceptually different layers of protocols for exchanging information over a network are described in the Open Systems Interconnection (OSI) Reference Model.

[0031] Communications between the network nodes are typically effected by exchanging discrete packets of data. Each packet typically comprises (1) header information associated with a particular protocol, and (2) payload information that follows the header information and contains information that may be processed independently of that particular protocol. In some protocols, the packet includes (3) trailer information following the payload and indicating the end of the payload information. The header includes information such as the source of the packet, its destination, the length of the payload, and other properties used by the protocol. Often, the data in the payload for the particular protocol includes a header and payload for a different protocol associated with a different, usually higher layer of the OSI Reference Model. The header for a particular protocol typically indicates a type for the next protocol contained in its payload. The higher layer protocol is said to be encapsulated in the lower layer protocol. The headers included in a packet traversing multiple heterogeneous networks, such as the Internet, typically include a physical (layer 1) header, a data-link (layer 2) header, an internetwork (layer 3) header and a transport (layer 4) header, and various application headers (layer 5, layer 6 and layer 7) as defined by the OSI Reference Model.

[0032] The client-server model of computer process interaction is widely known and used. According to the client-server model, a client process sends a message including a request to a server process, and the server process responds by providing a service. The server process may also return a message with a response to the client process. Often the client process and server process execute on different computer devices, called hosts, and communicate via a network using one or more protocols for network communications. The term "server" is conventionally used to refer to the process that provides the service, or the host computer on which the process operates. Similarly, the term "client" is conventionally used to refer to the process that makes the request, or the host computer on which the process operates. As used herein, the terms "client" and "server" refer to the processes, rather than the host computers, unless otherwise clear from the context. In addition, the process performed by a server can be broken up to run as multiple processes on multiple hosts (sometimes called tiers) for reasons that include reliability, scalability, and redundancy, among others. A well known client process available on most nodes connected to a communications network is a World Wide Web client (called a "web browser," or simply "browser") that interacts through messages formatted according to the hypertext transfer protocol (HTTP) with any of a large number of servers called World Wide Web servers that provide web pages. The services 110 are servers that use service data in service data structure 112a through 112n, collectively referenced hereinafter as service data structures 112.

[0033] In some embodiments, the UE 101 is a mobile terminal, as described in more detail below with reference to FIG. 8. In some of these embodiments, the mobile terminal has limited memory, computational resources, battery power or display area, or some combination. For such limited capacity mobile terminals, HTTP messages are reformatted by a mobile web service 130 and delivered to UE 101 in a more compatible protocol, such as the wireless application protocol (WAP).

[0034] In various embodiments, the UE 101 includes a browser 107, one or more applications available through the browser via common gateway interface (CGI) scripts (called CGI processes 109), or a standalone application 140, or one or more local data structures 142, such as one or more local databases, or some combination.

[0035] According to various embodiments, the system 100 includes an automated mashup service 120 either on the UE 101 or on a separate host connected to network 105, as shown in FIG. 1. The automated mashup service 120 uses data stored in a mashup data structure 122. In some embodiments, with the automated mashup service 120 on a separate host, a UE 101 communicates with the automated mashup service 120 through the browser 107. In some embodiments with the automated mashup service 120 on a separate host, a UE 101 communicates with the automated mashup service 120 through a separate mashup client 124 that performs one or more function not available in a standard browser. As used herein, the mashup client 124 refers to a separate application as shown in FIG. 1, or a browser displaying a web page provided by automated mashup service 120. The automated mashup service 120 forms a new source 128. The new source 128 may reside in mashup service 120, as shown, or on UE 101, or on one or more other hosts connected to network 105. In an illustrated embodiment, the automated mashup service 120 includes a local mashup tool 126 for forming a new source 128 that includes one or more sources on UE 101.

[0036] FIG. 2 is a diagram of the components of an automated mashup tool 200, according to one embodiment. By way of example, the automated mashup tool 200 includes a mashup GUI 203 and CGI processes 204 on mobile UE 202, a mobile web server 208, an automated mashup server 210 and mashup data 240. It is contemplated that the functions of these components may be combined in one or more components or performed by other components of equivalent functionality.

[0037] The mobile UE 202, such as UE 101, includes a mashup graphical user interface (GUI) 203 (e.g., either in a web page presented through browser 107 or as part of mashup client 124) and one or more CGI scripts 204 written in one or more scripting languages (such as PYTHON™ of the Apache Software Foundation of Delaware, United States). In some embodiments, the mobile UE 202 is a S60 mobile terminal from NOKIA™ Corporation of Espoo, Finland. The GUI 203 enables users of UE 202 to operate on the mashups. The GUI 203 is an example means for automatically presenting one or more extant sources and parameters and obtaining the advantage of easy and intuitive inclusion or one or more extant sources in a current mashup. In the illustrated embodiment, the CGI scripts include a calendar CGI 204a, a global positioning system (GPS) CGI 204b, a phonebook CGI 204c, and a media player CGI 204d, among others indicated by ellipsis. The CGIs are example means for obtaining the advantage of making sources on the user equipment available for the current mashup.

[0038] The mobile web server 208, such as mobile web server 130, communicates with the automated mashup server 210 using HTTP messages 209 and converts those messages to a mobile compatible format for exchanges with UE 202. By using a mobile web server 208, internal data on UE 202 can be accessed through HTTP request. For example, some CGI processes 204 are implemented as data providers for different data on UE 202. For example, in some embodiments, the mobile web server 208 is a S60 Mobile Web Server for the S60 dell phone mobile terminal from NOKIA™ Corporation of Esloo, Finland. The mobile web server is an example means for obtaining the advantage of identifying local sources on UE 202, such as CGIs 204. The mobile web server is also an example means for obtaining the advantage of allowing the automated mashup server 210 to exchange data in a generic HTTP format and hide the detaisl of exchanging data with different mobile devices, such as the S60 from NOKIA™.

[0039] The automated mashup service 210 includes a network manager module 212, a path recommender module 220 and a mashup execution engine 230. The new source 128 produced by executing the mashup execution engine 230 is depicted on server 210 for purposes of illustration. In other embodiments, the new source 128 is implemented on a different device connected to network 105, e.g., on UE 202. The network manager module 212 includes a local mashup tool 126 to identify and access one or more sources on UE 202, such as CGI sources 204 through mobile web server 208, for example. The local mashup tool is an example means for obtaining the advantage of including one or more local sources on user equipment 202 in the current mashup for the new source.

[0040] The network manager module 212 is configured to determine the network addresses, such as a uniform resource locator (URL) network address, of one or more sources on network 100, including one or more sources on UE 202 determined by local mashup tool 126. The network service manager module 212 is also configured to generate and format messages to be exchanged between the server 210 and other applications on the network (e.g., applications 110 on network 105 or browser 107 and CGI 109 or other application 140 on UE 101). In some embodiments, the network manger obtains descriptions of the network sources, such as metadata for the network sources. For example, in some embodiments, internal sources on UE 101 and internet sources such as applications 110 are accessed by a unified approach using HTTP (e.g. RESTful service).

[0041] The path recommender module 220 is configured to recommend APIs that can be combined in a new source based on semantic reasoning using text stored with the APIs or source metadata or based on histories of previously mashed sources or both, as described in more detail below. In various embodiments, the path recommender module 220 uploads the mashup history to a server and maintains a repository of APIs that have been mashed together in the past in mashup histories data structure 246, described below. The path recommender module is an example means for obtaining the advantage of an automated assist to a user for finding appropriate sources to include in the current mashup for the new source.

[0042] The mashup execution engine 230 is configured to interact with a user through GUI 203 to display network sources and recommended paths, e.g., in a graph as described in more detail below. The mashup execution engine is further configured to generate the mashup based on the graph to form new source 128. In some embodiments, the mashup execution engine 230 executes the mashup, e.g., hosts the new source 128. For example, in some embodiments, the mashup engine is developed as a plug-in in the S60 Mobile Web Server using PYTHON. The mashup execution engine is an example means for obtaining the advantage of automatically producing a new source from one or more different sources based on an intuitive graphical representation of the sources to connect. The automated mashup server 210 offers the advantage of offloading from a limited capacity mobile device to a more powerful remote device, computer resource intensive storage of all available sources and determination of relevance based on semantic and historical information. This allows a user to use a mobile device to mashup a new source.

[0043] The mashup data structure 240 includes a source metadata data structure 242, a public API data structure 244, and a mashup histories data structure 246. The source metadata data structure 242 stores data that indicates one or more sources on the network 105, such as a name of the source, uniform resource locator (URL) network address for the source, and a textual description of the source, such as its function and parameters or other items. The source metadata data structure 242 is therefore an example means for obtaining the inputs for semantic analysis of the source category and similarity to other sources and thus providing the advantage of automated semantic-based recommendation of relevant sources for the current mashup.

[0044] The public API data structure 244 stores data that indicates the APIs of one or more sources on the network 105 indentified by previous users of service 210,including each and every parameter name, parameter type and zero or more semantic labels provided by an owner of the API. The public API data structure 244 is therefore another example means for obtaining the inputs for semantic analysis of the source category and similarity to other sources and thus providing the advantage of automated semantic-based recommendation of relevant sources for the current mashup

[0045] The mashup histories data structure 246, also called a mashup repository) holds data that indicates extant sources that are used in previous mashups performed by users of the server 210. The mashup histories data structure 242 is therefore an example means for obtaining the probable utility of other sources and thus providing the advantage of automated probability-based recommendation of relevant sources for the current mashup.

[0046] Although modules and data structures are shown as integral blocks on particular network nodes or processes for purposes of illustration, in other embodiments one or more modules or data structures or portions thereof are arranged in different processes or nodes, or one or more are omitted, or one or more other modules or data structures are included.

[0047] FIG. 3A is a diagram of an application programming interface (API) data structure 300, according to one embodiment. Data structure 300 is a particular embodiment of public API data structure 244. The API data structure 300 includes an API field 310 along with zero or more other API fields indicated by ellipsis, collectively referenced hereinafter as API fields 310. Each API field 310 holds data for one API encountered by automated mashup server 210.

[0048] In the illustrated embodiment, each API field 310 includes an API identifier (ID) field 312, a network address field 314 for the API and a type/text field 316. Each API field 310 also includes a parameter field 320a along with zero or more other parameter fields indicated by parameter field 320b and ellipsis, collectively referenced hereinafter as parameter fields 320. In the illustrated embodiment, each API field 310 also includes a semantic label field 322a along with zero or more other semantic label fields indicated by semantic label field 322b and ellipsis, collectively referenced hereinafter as semantic label fields 322. Each semantic label field 322 corresponds to a respective parameter field 320. In some embodiments, semantic label fields 322 are omitted.

[0049] The API ID field 312 holds data that indicates an identifier for the API, such as a name of a network application 110 to which the API pertains. The network address field 314 holds data that indicates where the API can be accessed on a network, such as on network 105. For example, in some embodiments, the network address field 314 holds data that indicates an Internet Protocol (IP) address and Transmission Control Protocol (TCP) port number. In some embodiments, the network address field 314 holds data that indicates a URL name or other name resolved by a network name server.

[0050] The type/text field 316 holds text data that indicates the type or description for the API, e.g., data indicating a search engine or a map application or any text provided at the API network address. This field is used in some embodiments to determine a semantic class or context or set of concepts to associate with the API for determining automatically whether the API can be used in the new source or is compatible with the item or parameter of another extant source, as described in more detail below.

[0051] The parameter field 320 holds data that indicates each parameter of the API, such as the parameter name, the data type (e.g., numeral, decimal, text string, character set, date/time, etc.), an operation associated with the parameter, and whether input or output or both.

[0052] The semantic label field 322 holds data provided by an owner of the API to allow a semantics engine to determine a semantic class or context or set of concepts for the API parameter. In some embodiments, a user is prompted for the semantic label when the API is generated, e.g., using mashup service 210 and GUI, as described in more detail below. For example, an input parameter may be named "Date," but the semantic label will include data that indicates a semantic context or clarification, such as a date of hire or a birth date. The semantic label fields are example means to provide semantic information to obtain the advantage of an automated recommendation of relevant sources or parameters or both. In some embodiments, the semantic label fields 322 are omitted.

[0053] Although fields are shown in FIG. 3A, and in subsequent figures 3B and 3C, as integral blocks in a particular order on a single portion of memory for purposes of illustration, in other embodiments one or more fields or portions thereof are arranged in different order on one or more portions of memory, or one or more databases, or one or more fields or portions thereof are omitted, or one or more other fields are added. [0054] Sources are not confined to APIs, and some sources are data structure or web pages with certain items in particular fields. Thus a source data structure (not shown) is analogous to the API data structure shown in FIG. 3 A, but with item fields replacing parameter fields. Metadata about all sources, including non-API sources, are also useful for automatically determining what sources are available for mashup. As used herein, a source refers to any data or functionality available on the network, whether an API is available or not, and items refer to individually addressable portions of the source, whether an API parameter or not.

[0055] FIG. 3B is a diagram of a source metadata data structure 330, according to one embodiment. Data structure 330 is a particular embodiment of source metadata data structure 242 depicted in FIG. 2. Data structure 330 is therefore an example means for obtaining the advantage of semantic-based recommendation of relevant sources or items within the source. Here, metadata is actually encoded in an Ontology language, e.g. a W3C standard language RDF Schema (available in directory rdf-schema in subdirectory TR at the W3 domain). In the syntax of RDF Schema, a piece of metadata is presented as a RDF class with an ID and a URL. In addition, it includes a free text to describe the class. A class has one or more properties and the value of each property is an object, which is actually the ID or URL of another RDF class.

[0056] The source metadata data structure 330 includes a source field 340 along with zero or more other source fields indicated by ellipsis, collectively referenced hereinafter as source fields 340. Each source field 340 holds metadata for one source encountered by automated mashup server 210.

[0057] In the illustrated embodiment, each source field 340 includes a class identifier (ID) field 342, a network address field 344 for the source and a type/text field 346. Each source field 340 also includes an property field 350a along with zero or more other property fields indicated by property field 350b and ellipsis, collectively referenced hereinafter as property fields 350. The properties refer to the portions of a web page or data structure, such as a schema, a header, or a field in a form or data record. In the illustrated embodiment, each source field 340 also includes an object field 352a along with zero or more other object fields indicated by object field 352b and ellipsis, collectively referenced hereinafter as object fields 352. Each object field 352 corresponds to a respective proeprty field 350.

[0058] The class ID field 342 holds data that indicates an identifier for the class of the source. The network address field 344 holds data that indicates where the class can be accessed on a network, such as on network 105. For example, in various embodiments, the network address field 344 holds data that indicates a URL for the RDF Schema class.

[0059] The type/text field 346 holds text data that indicates the type or description for the source, e.g., a blog, newsfeed, social network home page, or any text provided at the source network address, including a name of the network application 110 or local application 140 to which the source pertains. This field is used in some embodiments to determine a set of concepts to associate with the source for determining automatically whether the API can be used in the new source or is compatible with the item or parameter of another extant source, as described in more detail below. Thus the type/text field 346 is an example means for achieving the advantage of semantic-based recommendation of relevant sources and items.

[0060] Each property field 350 holds data that indicates each item of the source, such as the item name, the data type (e.g., image, sound clip, numeral, decimal, text string, character set, date/time, etc.), an operation associated with the item, and whether the item is a constant or variable or input or output. Thus property field 350 is an example means for obtaining the advantage of accessing portions of any source on a network, whether an API is available or known or not.

[0061] The object field 352 holds data that indicates a semantic value for the property, such as the ID or URL of another RDF class. This allows a semantics engine to determine a semantic concept or context for the property. Therefore object field 352 is an example means for obtaining the advantage of a semantic-based recommendation of relevant sources and items.

[0062] FIG. 3C is a diagram of mashup histories data structure 360, according to one embodiment. Data structure 360 is a particular embodiment of mashup histories data structure 246, depicted in FIG. 2. The mashup histories data structure 360 includes a mashup field 370 along with zero or more other mashup fields indicated by ellipsis, collectively referenced hereinafter as mashup fields 340. Each mashup field 370 holds data for one previous mashup determined by automated mashup server 210. In various embodiments the information in the mashup histories data structure provides evidence of sources that are compatible with each other, because they are known to have been used together in a prior mashup. Thus mashup histories data structure 360 is an example means for achieving the advantages of determining source compatibility and also providing probability-based recommendation of relevant sources and items.

[0063] In the illustrated embodiment, each mashup field 370 includes a mashup identifier (ID) field 372, a network address field 374 for the source formed by the mashup, a mashup descriptive text field 376, and a mashup owner field 378. Each mashup field 370 also includes a source field 380a along with zero or more other source fields indicated by source field 380b and ellipsis, collectively referenced hereinafter as source fields 380. In the illustrated embodiment, each mashup field 370 also includes an items field 382a along with zero or more other items fields indicated by items field 382b and ellipsis, collectively referenced hereinafter as items fields 352. Each items field 382 corresponds to a respective source field 380. In some embodiments, items fields 382 are omitted.

[0064] The mashup ID field 342 holds data that indicates an identifier for the mashup, such as a name of new source generated by the mashup. The network address field 374 holds data that indicates where the new source can be accessed on a network, such as network 10, e.g., a TCP IP address and port or URL or other name resolved by a network name server.

[0065] The mashup descriptive text field 376 holds text data associated with the mashup, such as search terms, operations and choices mad while the mashup was generated by a user. This data is used in some embodiments to determine a semantic context for the mashup and reasoning for selecting certain sources over others. Thus field 376 is an example means for achieving the advantage of semantic-based recommendation of relevant mashups to include in a current mashup.

[0066] The mashup owner field 378 holds data that indicates a user of the mashup service who formed the mashup indicated in field 372. In some embodiments, data in this field is used to give more weight for relevance to compatible sources used by a current user than to compatible sources used by different users of the mashup server 210. Thus field 378 is an example means for achieving the advantage of determining relevance of a source to a particular user, to make more useful automated recommendations.

[0067] The source field 380 holds data that indicates each source included in the prior mashup indicated in field 372, such as the name or network address of the source. The items field 382 holds data that indicates which parameters or items of the source in the corresponding source field 380 were used in the mashup. In some embodiments, the items field 382 also includes data that indicates which items or parameters were used an input and which as output. In some embodiments, the items field 382 also includes data that indicates a parameter of another source that was associated with a particular parameter. For example, a particular output item in items field 382a associated with source 380a is flagged with a number or label, e.g., "A." and a different input item in items field 382b associated with source 380b is flagged with the same number or label, e.g., "A." Thus, in this example embodiment, the items fields 382 indicate that the particular output item of one source served as the input item of the different source. In some embodiments, such information is used to suggest or recommend a particular source item to a current user of the mashup server 210. Therefore, items field 382 with flags that connect different items is an example means of obtaining the advantage of identifying individual items used in the prior mashups and determining the compatibility, probability or semantic relevance or some combiantion, at the level of individual items in the sources.

[0068] FIG. 4A is a flowchart of a process 400 for a mashup service to support an automated mashup tool, according to one embodiment. In one embodiment, a local mashup server performs the process 400 and is implemented on UE 101 in, for instance, a chip set including a processor and a memory as shown FIG. 7, or mobile terminal as shown in FIG. 8, or on a general purpose computer shown in FIG. 6. In another embodiment, the remote mashup server 210 performs the process 400 and some or all the components of mashup server 210 are implemented as automated mashup service 120 on a different network node in, for instance, a chip set including a processor and a memory as shown FIG. 7, or on a general purpose computer shown in FIG. 6. [0069] Although steps are shown as integral entities in a particular order in FIG. 4A and subsequent flow chart 4B, for purposes of illustration; in other embodiments, one or more steps or portions thereof are performed in a different order, or overlapping in time, in series or in parallel, or one or more steps or portions are omitted or one or more other steps are added, or the method is changed in some combination of ways.

[0070] In step 401, a request is received from user equipment to mashup extant sources in a current mashup to form a new source. Any module or process on the UE may send the request. For example, in some embodiments, the request is sent in response to a user selecting a link in a web page sent from remote automated mashup server 210 as an HTTP message 209 to mobile web server 208 which forwards the link in a reduced web page sent in a WAP message to a mobile device UE 202. In some embodiments, the request is sent from a mashup client 124 on UE 101. In some embodiments, the mashup client 124 and server 120 are both on UE 101, and the request is sent from one component or module providing a GUI to another component or module acting as automated mashup server 210. In some embodiments, the request includes an identifier or security credential for a user of UE 101, e.g., after a user logon process well known in the art.

[0071] In step 403 the local sources on the user equipment are determined. Any method may be used to determine the local sources. Internal data on user equipment and internet data can be accessed by a unified approach using HTTP (e.g. RESTful service). In some embodiments, the mashup client 124 polls the CGI applications registered with the browser 107 or prompts a user of UE 101 to identify one or more other sources, such as application 140 or local data structure 142, or some combination. A number of CGIs have been developed under S60 Mobile Web Server for the S60 mobile devices from NOKIA™, using PYTHON™. In some embodiments, a script sent to browser 107 by automated mashup service polls the CGI applications registered with the browser 107 or prompts the user to identify one or more other sources. In some embodiments, using the mobile web server 208, internal data on user equipment can be accessed through an HTTP request to a browser 107 or mashup client 124. One or more CGIs 109 are implemented as data providers for different local data 142 on UE 101. Data indicating the local sources are sent to a remote mashup service 120 in one or more messages, e.g., WAP messages to mobile web server 208 which are converted to HTTP messages 209 sent to automated mashup server 210.

[0072] Step 403 includes storing metadata about the local sources in mashup data structure 122, e.g., in source metadata data structure 242, such as in source ID field 342, network address field 344, type/text field 346, and one or more property fields 350 of source metadata data structure 330. Thus step 403 is an example means for obtaining the advantage of including local sources among sources used in a current mashup to generate a new source for the network.

[0073] In step 405, semantic labels for local sources or parameters or both are determined. Any method may be used to obtain the semantic labels. For example, in some embodiments a user is prompted via mashup GUI 203 for semantic labels for one or more parameters or other source items in HTTP messages sent to browsers 107, e.g., through mobile web server 208, or to mashup client 124. In some embodiments, text surrounding the item is automatically selected to use as a semantic label for the item. The text is associated with a concept in the semantic vocabulary for each item or property, such as a RDF Schema URL for the concept. Step 405 includes storing semantic objects in mashup data structure 122, e.g., in source metadata data structure 242, such as in one or more objects fields 352 of source metadata data structure 330. Therefore, step 405 is an example means for achieving the advantage of semantic-based recommendations of relevant sources and items.

[0074] In step 407, the most relevant sources on the network, including their compatibility, i.e., their ability to be used together in a mashup, are determined for the current mashup. The compatibility and other contributions to relevance are dependent on how much is known about the current mashup by the user for producing the new source. In some embodiments compatibility is based at least in part on histories of previous mashups. In some embodiments, one or more local sources are included or assumed to be included. In some embodiments, semantic similarity is also determined using a semantics engine and text associated with sources or associated with parameters or other items of the sources. A method for determining compatibility, semantic similarity and relevance is described in more detail below with respect to FIG. 4B. In some embodiments the most relevant sources or items are determined based on a threshold measure of relevance. In some embodiments, the most relevant are a limited to a number M that can readily be displayed on the user equipment. In some embodiments, step 407 is performed by the path recommender module 220. Thus step 407 is an example means for obtaining the advantage of automatically determined relevance for one or more sources or items.

[0075] In step 409, a presentation of a graph is caused on the user equipment to serve as a GUI. The graph includes one or more icons representing sources and one or more edges representing selected or compatible sources. As used herein, an icon is a set of picture elements, called pixels, for presentation on a video display device, with which one or more actions are associated based on user operation of a pointing device co-located with the icon. In some embodiments only the M most relevant sources are included in the graph or one or more sources are recommended among those displayed. The GUI enables users to operate on the sources during the mashup to generate the new source. In various embodiments, a user mashups one or more sources for the new source by choosing to add or remove edges between icons representing those sources. Thus step 409 is an example means of achieving the advantage of providing a user with an intuitive way to indicate sources or items to include in the current mashup and automatically assist with one or more recommendations and information about relevance and compatibility.

[0076] Any method may be used to cause the GUI with the graph to be presented on the user equipment. For example, during step 409, a GUI web page is generated on remote service 120 and sent to the browser 107, where the web page elements constitute mashup GUI 203. In some embodiments, data representing the most relevant sources and compatibilities are sent to a mashup client that includes and generates the mashup GUI based on the sent data. In an example embodiment, the GUI is developed using FLASHLITE™ by ADOBE™ of San Jose California, running on S60 mobile devices from NOKIA™ .

[0077] FIGs. 5A-5F are diagrams of user interfaces utilized in the processes of FIG. 4A, according to various embodiments. FIG. 5F is a diagram that illustrates a portion 520e of a mashup GUI using a hierarchical list instead of a graph of icons. This portion 520e includes a show block list button 591, a collapse button 592 and a run button 593. The hierarchical list is presented as indented folders for each successive level of the hierarchy, all associated with categories of sources indicated by their URLs, with one or more parameters as individual entries at the deepest level of the hierarchy. For example, the root of the hierarchy is a class of semantic categories represented by URL 581. The next level is the category of sources with URL 582. Two relevant subcategories are device sources with URL 583a and network sources with URL 583b in the third level of the hierarchy. In the fourth level of the hierarchy are four sources identified by URL 585a, URL 585b, URL 585c, and URL 585d. URL 585a is a phonebook CGI under device source ( URL 583a); while URL 585b is a set of social network sources, URL 585c is a set of music sources and URL 585d is a set of map sources under network sources (URL 583b). Phonebook URL 585a includes four parameters 586a through 586d, e.g., name, email, instant message identifier (ID) and cell phone number. Similarly, social network URL 585b includes four sources at the next level of the hierarchy 586e through 586h for social network sources A through D; URL 585c for music sources includes four sources 586i through 5861 for music stores A through D, respectively. URL 585d for maps includes one source, maps A with URL 586m.

[0078] The show block list button 591 is activated to toggle between the list and no list. The collapse button 592 is activated to hide the details below a selected level of the hierarchy. The run button 593 is an alternative embodiment of the generate button 529 described below.

[0079] FIG. 5A is a diagram that illustrates an example graphical user interface (GUI) 520 on display 501 of user equipment, e.g., mobile terminal 800 described in more detail below with reference to FIG. 8. The display 501 includes a screen header area 510 that presents graphical information for the user equipment, such as wireless service provider name and logo, signal strength and battery life. The display 501 includes other screen areas 512 that presents other graphical information, e.g. for one or more other applications concurrently running on the user equipment. The display 501 includes an example mashup GUI screen area 520.

[0080] In the illustrated embodiment, the mashup GUI screen area 520 includes a new source name area 522, and recommend button 524, list/icon button 526, save button 528 and generate button 529, collectively referenced hereinafter as buttons 523. The mashup GUI also includes icons 530 each representing a different source and edges 534 connecting sources that are compatible, thus forming a graph of sources available for mashup. Icons 530 include one or more icons 532 representing one or more local sources on the user equipment. In the illustrated embodiment, the mashup GUI screen area includes scroll controller 538 to scroll off screen portions of the graph into a limited view area. Thus GUI screen area 520 is an example means for providing the advantage of controlling the mashup at user equipment.

[0081] The new source name area 522 presents data indicating a name for the new source being generated by the current mashup. The buttons 523 are activated based on user operation of a pointing device for the user equipment, such as a touch screen or a cursor 513 controlled by one or more buttons, slide, wheel or mouse. The recommend button 524 is activated to toggle between displaying and hiding one or more sources or edges automatically recommended by the path recommender module of automated mashup server 210. The list/icon button is activated to toggle between displaying compatible sources as icons in a graph, as depicted in FIG. 5A or as a hierarchical list as described in more detail below with reference to FIG. 5F. The save button is activated to save the current version of the mashup for further editing at a later time. The generate button 529 is activated to generate the new source by mashing up the sources represented by icons currently connected by the user, e.g., by invoking the mashup execution engine 230.

[0082] According to some embodiments, the icons 530 included in the graph represent the M most relevant icons that are compatible with a particular source. For user equipment that is a mobile device with limited display size, M is small, e.g., about 10. For user equipment that has a more extensive display, M is large, e.g., about 50. Thus GUI screen area 520 is an example means for providing the advantage of controlling the mashup even at mobile user equipment.

[0083] In some embodiments, the GUI is further designed to assist the user in determining which sources to include in the mashup. For example, the particular source selected by the user, e.g., from the hierarchical list in FIG. 5F, is placed at the center of the display, and called the central source hereinafter. Only sources compatible with the central source are included, as indicated by dashed edges 534 extending from each icon to the icon at the center. A source is compatible with another if it has at least one parameter or item that can be used as a parameter or item for another source. In the illustrated embodiment, compatibility is determined based on use together in one or more prior mashups stored in the mashup histories data structure 246. Of all the compatible sources for the central source, only the M most relevant are included in the graph. Relevance includes the semantic similarity or frequency of prior uses or some combination, as described in more detail below. In the illustrated embodiment, the icons are distributed in the graph so that distance between the icon and the central icon is proportional to the relevance of the icon; the more relevant the icon to the central source the shorter the distance to the central icon. In some embodiments, the size of the icon is related to the frequency of prior use of the source in other mashups; the more frequent the prior use of the icon, the larger the icon. In some embodiments, the color of the icon indicates the type of source, such as determined by the semantic class e.g., search engine, dictionary, mapping application, social network, music source, etc. Thus the graph of mashup GUI is an example means of achieving the advantage of intuitively presenting candidate sources for the current mashup.

[0084] In some embodiments, the local icons 532 for sources on the user equipment are segregated on one side. In some embodiments, one of the local icons is placed at the center. In some embodiments, the local icons are integrated with the other icons. In some embodiments, the local icons are given a separate color or border to highlight them within the graph. Thus the graph of mashup GUI is an example means of achieving the advantage of intuitively presenting candidate local sources as well as candidate network sources for the current mashup

[0085] Thus, in some embodiments, during step 409, the mashup GUI with graph of connected icons is caused to be presented on the user equipment, and step 409 is an example means of achieving the advantages of the graph for indicating candidate sources for a current mashup.

[0086] In step 411, it is determined whether input has been received from a user indicating a particular source. For example, it is determined whether a user has operated a touch screen or other pointing device, e.g., controlling a cursor, to be positioned over a particular icon. If so, then in step 413 parameters or other items associated with the API or other source, respectively, is caused to be presented. For example, the mashup client presents the parameters or other items, or the remote automated mashup server is notified of the pointing device operation and, in response, sends one or more popup windows with one or more descriptions of the source.

[0087] Figure 5B is a diagram that illustrates an example mashup GUI screen area 520a on display 502 of user equipment, when a user has selected a source icon. The display 502 includes a screen header area 510 and other screen areas 512 as described above, and modified mashup GUI screen area 520a. The buttons 523 are as described above for FIG. 5A. The icons 530 including local icons 532 and edges 534 are as defined above. The illustrated difference includes the cursor 513 moved over a source icon to indicate that icon by a user employing a pointing device. As a consequence, a screen area 550 is presented with a summary description of the source associated with the icon being pointed to. The area 550 is presented over the graph, obscuring any icons or edges that were in the same location. In some embodiments, the area 550 appears to be partially translucent so that one or more underlying icons or edges or some combination are partially evident. Presented in the description of source area 550 is text describing the source and the functions or data that the source provides.

[0088] Also shown in mashup GUI screen area 520a is an area 560 where is presented graphical representation of an extant mashup using that source from the mashup histories data. This is useful in guiding a user to make a selection of a source function or item, and benefits the user from the experience of other users or reminds the user of the user's own prior mashups. Thus GUI screen area 560 is an example means for achieving the advantage of automatically suggesting viable choices for one or more sources to include in the current mashup. Also depicted in FIG. 5B is the removal of one or more icons or edges; since the user selection of the source under cursor 513 changes the determination and ranking of relevance of the other sources. Thus, the system can learn the user's intention by analyzing the traces and recommend the most relevant APIs. Thus removing less relevant icons is an example means of providing the advantage of learning user intent based on user actions to recommend more relevant sources or items.

[0089] If the user further selects the source, e.g., by touching it a second time or depressing a soft key or other key, then the user is presented with one or more options for using the source in a mashup. Figure 5C is a diagram that illustrates an example mashup GUI screen area 520b on display 503 of user equipment, when a user has selected a source icon for inclusion. The display 503 includes a screen header area 510 and other screen areas 512 as described above, and modified mashup GUI 520ab. The buttons 523 are as described above for FIG. 5 A. The icons 530, including local icons 532, and edges 534 are as defined above. The illustrated difference includes mashup options area 540 presented over the graph. In the mashup options area 550 is the information describing the source, such as functionality/operations, type of input, and suggested sources and items to serve as input or output of the source. For example, the source name is presented in source name area 542 of the mashup options area 540; the source operations are presented in the operation area 544; and the parameters or other items are presented in the parameters area 546 of the mashup options area 540. Thus mashup options area is an example means to achieve the advantage of an automated assist in receiving user input indicating sources and items to include in the current mashup.

[0090] In the illustrated embodiment, the source name area 542 presents data indicating a name for the extant source to be included in the current mashup. The operation area 544 presents a list of one or more operations that can be performed by the source, e.g., in a pull down menu. The parameters area 546 presents a list of the parameters or other items. In some embodiments, if one or more items are associated semantically or by mashup history with an item of the central source or some other source already included in the current mashup, that item or those items are highlighted, e.g., by presenting them first in a list, or by filling a background with a particular color, or some combination. Thus parameters area 546 is an example means of providing the advantage of automatically recommending one or more items of a source for inclusion in a current mashup.

[0091] In step 415, it is determined whether input has been received from a user indicating a particular API parameter or other source item. For example, it is determined whether a user has operated a touch screen or other pointing device, e.g., controlling a cursor, to be positioned over a particular item listed in area 546. If so, then in step 417, the selected parameters or other item associated with the API or other source, respectively, is caused to be added to the current mashup for the new source. Control passes back to step 407, described above, to recomputed relevance based on the selection made by the user.

[0092] In some embodiments, the operation area 544 also includes one or more other aspects of the operation, such as whether an item is to be used as input or output, a functionality (e.g., an source operation) in which to use the item. For example, a functionality of a mapping source is to "get a geo-tagged photo" which means to get an image file associated with geographic coordinates, such as latitude and longitude coordinates from a GPS. If that functionality is selected then, it is assumed for purposes of illustration, that an input for the operation is one or more geographic coordinates and a category of the photo, such as a building, a person, a municipal service, traffic, weather, etc. In some embodiments, the mashup options area 540 includes in parameters area 546 a list of items or parameters of the most relevant compatible sources that can provide geographic coordinates or photo categories, for example in order of relevance ranking. The user need only select one of the suggested items to serve as input to the function for the source.

[0093] Thus, during step 417 a parameter of an extant source is added to the mashup for a new source. In some embodiments, the parameter is from a local source. In some embodiments, the direction of parameter use as output from one source to input into another source, is indicated by a direction of the edge between the two icons representing the sources. Control passes back to step 407, described above, to recomputed relevance based on the selection made by the user. Thus step 417 is an example means for achieving the advantage of providing an automated assist for connecting one source to another in the current mashup.

[0094] Figure 5D is a diagram that illustrates an example GUI screen area 520c on display 504 of user equipment, when a user has selected a parameter of a source icon. The display 502 includes a screen header area 510 and other screen areas 512 as described above, and modified mashup GUI 520c. The buttons 523 are as described above for FIG. 5A. The icons 530 including local icons 532 and edges 534 are as defined above. The illustrated difference includes the cursor 513 moved away from a source after a parameter and operation selection has been made. As a consequence of the selected operation and items from existing sources, a solid, directional edge 570 is inserted into the graph. In the illustrated embodiment, an output item from one of the local sources represented by local icons 532 is used as an input to an extant source, as indicated by the solid directional edge 570. Thus solid edge 570 is an example means for achieving the advantage of intuitively representing selected connections among extant sources for the current mashup.

[0095] Returning to FIG. 4A, in step 419, it is determined whether input has been received from a user indicating a particular operation is to be added to an instruction set for the new source, such as a mathematical, logical, string, branch or looping operation. If so, then in step 421, the particular operation is added to an instructions set for the new source. For example, a text box is presented in a mashup GUI, not shown, in which the user can insert one or more instructions for operating on data already available in the mashup, as is well known in the art, without involving another extant source.

[0096] In step 423, it is determined whether input has been received from a user indicating that the mashup is complete. For example, based on a completed graph with solid directional edges, a user determines that a new source appears to be complete, and activates the generate button 529.

[0097] Figure 5E is a diagram that illustrates an example GUI screen area 520d on display 505 of user equipment, when a user has added several edges to the graph. The display 505 includes a screen header area 510 and other screen areas 512 as described above, and modified mashup GUI 520d. The buttons 523 are as described above for FIG. 5A. The icons 530 including local icons 532 and edges 534 are as defined above. As a consequence of the selected operation and items from existing sources, multiple solid, directional edges 570, 570a, 570b and 570c are included in the graph.

[0098] An advantage of the graph in mashup GUI is that users can surf the data and API network just like surfing the Web; and interested APIs of interest can be mashed up easily just by leaving a trace through the graph in the GUI. The graph with directional edges representing chosen connections is an example means for achieving this advantage. [0099] If the mashup is complete, as determined in step 423, then, in step 425, the new source is generated based on the parameters and other items of extant network or local sources and other instructions and operations indicated. In some embodiments, step 425 is performed by the mashup execution engine 230. For example, in some embodiments, the mashup engine is developed as a plug-in in the S60 Mobile Web Server using PYTHON. The mashup engine is able to execute the generated mashup graphs, and present the mashup result by integrating data from different sources.

[0100] In some embodiments, the available sources and items or the selected sources and items are presented in list form instead of graph form. For example, in some embodiments using mobile devices of small screen areas, a list form is preferred. In the illustrated embodiment, the list/icon button 526 is used to toggle a display between a graph of icons to a hierarchical list.

[0101] FIG. 4B is a flowchart of a process 440 for a path recommender module to support an automated mashup tool, according to one embodiment. In step 441, a request is received for relevant sources and edges for a current mashup to produce a new source for the network. In some embodiments, the request includes data indicating a particular semantic context for the new source or a particular extant source that is used in the mashup, such as a particular local source.

[0102] In step 443, text associated with one or more sources and parameters and the current mashup are determined. For example, text associated with the type/text field 316 and semantic label fields 322 of one or more APIs, or text associated with type/text field 346 and semantic label fields 352 for other sources, or some combination are fed to a semantics engine to perform semantic analysis and determine a set of one or more semantic concepts for the new source. In some embodiments, a particular API or source is indicated (e.g., in the request received in step 441) to narrow the semantic analysis to one or a few of the local or network sources. In other embodiments, no API or source is indicated, and one or more local APIS or local sources, or some combination, are mined for text to feed to the semantics engine.

[0103] In step 445 a semantic class (also called a category herein) is determined for the current mashup based on the semantic concepts and contexts of the text provided during step 443. Any semantics module may be used to determine the semantic concepts or context of the text determined. In some embodiments a semantic web technology is employed, well known in the art, such as Resource Description Framework (RDF) Schema. RDF Schema is an extensible knowledge representation language, providing basic elements for the description of vocabularies (called ontologies) intended to structure RDF resources. RDF Schema is published by the World- Wide Web Consortium (W3C). In some embodiments, the metadata of APIs in the public API data structure 244 are input to RDF Schema to generate a vocabulary of source types, operations and parameters.

[0104] At the start, there are no parameters or sources already in the current mashup, so the semantic context of the current mashup is null, in some embodiments, and semantic context is assumed based on one or more of the sources identified above. If one or more sources or parameters are already included in the current mashup, then the text fields associated with those sources are also used to determine the semantics of the current mashup. If no sources are identified, and no text is provided in the request 441, and no mashup is yet started, then the semantic context of the current mashup is null. Otherwise the text available for the current mashup is analyzed using the RDF Schema to determine one or more semantic concepts for the mashup and a semantic class (e.g., a category) for the mashup based on the one or more concepts.

[0105] In step 447, one or more network sources in the same class or classes as the current mashup are determined. Such sources are considered semantically relevant to the current mashup. If the semantic class of the current mashup is null, e.g., at the start, then step 447 is omitted. In some embodiments, the semantics engine is not confined to determining one or a few classes, but instead is used to determine a measure of semantic similarity also during step 447. For example, the multiple semantic concepts of varying degrees of confidence are determined for each source, based for example on the descriptive text and one or more parameters or other items and their semantic labels, if any. A similarity measure is based on comparing the array of concepts and confidences of two sources. The more concepts and confidences match, the higher the similarity measure. In some embodiments, a similarity measure is determined between each parameter in the current mashup and each parameter in the extant sources. In some of these embodiments, the similarity of the source is a function of the similarities of the items included in the source. Step 447 is an example means to achieve the advantage of automatically determining which items and sources are potentially relevant for the current mashup.

[0106] In step 449 an entirely independent assessment of various sources is determined, based on probability computed using the mashup histories stored in mashup histories data structure 246. At the start, when no sources have yet been included in the current mashup, the most probable source is the source that is used most in the mashup histories. As various sources are included in the mashup for the new source, conditional probabilities based on the coincident occurrence of a source with the sources already included in the current mashup. In some embodiments, the probability of a source being used in the current mashup is given by Equation la, where s is the number of sources already included in the current mashup (e.g., already connected in the graph of the mashup GUI by a solid directional edge), A is a candidate source, and Bi for i=l to s are the s sources already included in the current mashup. The probability that A will be added to the mashup, represented by Pincluded(A), is:

Pincluded(A) = NA / N, if s=0

= P(A | Ui=l,s Bi), if s>0 (la)

Where N is the number of mashups fields 370 in the mashup histories data structure 360 and NA is the number of mashup fields 370 for which A is included in a source field 380. P(A|X) indicates the conditional probability that A occurs given the condition X. In equation la the condition X is Ui=l,s Bi, which indicates the union of all occurrences of the Bi for i=l to s. In some embodiments, the conditional probability is computed using equation lb.

P(A | Ui=l,s Bi) = ∑ (-l)k-l ∑ P(A | BI) (lb) k=l,s I (I is a subset of (1,... , s) of size k) which indicates that the conditional probability is based on the probability that A is included in a mashup given that any set of I Bi up to all s Bi are included in a mashup based on mashup histories.

[0107] If A has been used with B in the mashup histories then A and B are compatible and can be connected by a dashed line in the graph In some embodiments, step 449 includes determining the probability of each parameter or item of a source based on items already included and the histories. Thus, step 449 is an example means to achieve the advantage of automatically determining which items and sources are compatible and probable for inclusion in the current mashup.

[0108] In step 451 the relevance of each source is determined based on the semantic class or similarity or the probability based on the histories of past mashups by others or the user or some combination. For example, the relevance of each source is determined as the probability of occurrence computed based on Equation la and lb times the semantic similarity times a factor greater than one if the user of user equipment has used the source in a prior mashup. If only semantic classes are used, then, in some embodiments, the semantic similarity is 1 if the source is in the same class as the current mashup and zero if the source is in a different semantic class and 0.5 if the class of the current mashup is not known (e.g., at the start). In some embodiments, the relevance of each parameter or other item of a source is determined, and the relevance of the source is determined by adding the relevance of each parameter or other item in the source. Thus step 451 is an example means for achieving the advantage of automatically combining semantic similarity, compatibility and probability to determine an overall relevance of extant sources or items, allowing a user to perform a mashup with less prior knowledge than is required in other approaches.

[0109] In step 453 the sources are ranked based on the relevance measure computed in step 451. In some embodiments, the most relevant source is placed at the center of the graph for the mashup GUI. In some embodiments, the current mashup is placed at the center of the graph for the mashup GUI. Thus step 453 is an example means for achieving the advantage of automatically forming a graph that intuitively depicts relevance.

[0110] In step 455 edges are determined between sources based on semantics and histories. For example, all sources that have been mashed up in the histories are connected by dashed line segments as edges. In some embodiments, directions of mashups are indicated on the edges, such as with arrows. Thus step 455 is an example means for achieving the advantage of automatically forming a graph that intuitively depicts compatibility. [0111] In step 457, the M most relevant sources and edges are returned with their relevance rank, probability of occurrence and semantic class. In some embodiments the graph icons presented in later steps of process 400 reflect these properties, e.g., the distance to a source depends on the relevance, the size of the icon depend on the probability of occurrence, and the color of the icon depends on the semantic class. Thus step 457 is an example means for achieving the advantage of automatically forming a graph that intuitively depicts compatibility, relevance and type of candidate sources.

[0112] Thus a user can readily mashup local and network sources with a simple interface that can be adapted to the small screen size, memory and battery life of a mobile device, such as a cell phone. For example, it is assumed for purposes of illustration that on-device sources include GPS, MediaP layer, Calendar and Phonebook as shown in FIG. 2, all implemented as CGIs of a Mobile Web Server 208 and accessed through HTTP RESTful request messages 209. These are presented in a block list as shown in FIG. 5F. It is further assumed that the user selects the phonebook service from the list in FIG 5F, and drags the icon to the central icon position of screen area 520. It is further assumed that the user indicates the icon with a pointing device, then clicks the "RECOMMEND" button 524.

[0113] The recommended sources are shown in the screen area 520 as icons 530 and they are connected to the phonebook service at the center by edges 534. It is further assumed that the recommended sources 530 include a social network B service (URL 586f) that can show a person's current status given an email address, a maps A service (URL 586) that can shown a person's address on a map, and a phone lookup service (not shown) that can show the region (e.g. a state) where a phone number is registered. The inputs of all recommended services are outputs of the phonebook service at URL 585a, and thus edges 534 can point outward from the central icon. The distances form the recommended icons to the selected source at the center are different. The shorter the distance, the more relevance between the recommended source and the selected source. The relevance is calculated by a number of factors like the history of other's mashups and the current user's preference as presented above. [0114] If there is no recommendation, or the user isn't satisfied with the recommendation, the user selects an arbitrary source from the list (e.g., in FIG. 5F) and adds it to the screen. Then the can connect two sources manually, presented for example as a solid directional edge 570. The user continues with zero or more additional sources and connections.

[0115] Once finished, the user clicks the "GENERATE" button 528 or "RUN" button 593. During the generation, e.g., by the mashup execution engine 230, the mashup GUI shows the mashup by retrieving data from different sources, controlling the execution order, enabling debugging from step to step and showing the final result on a popup window. Then the mashup, e.g., the new source, is uploaded to a mashup server on Web, and the data for the new source is stored as mashup history, e.g., in data structure 246. The mashup histories data is used to guide other users for subsequent mashups.

[0116] The processes described herein for providing an automated mashup tool may be advantageously implemented via software, hardware (e.g., general processor, Digital Signal Processing (DSP) chip, an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Arrays (FPGAs), etc.), firmware or a combination thereof. Such exemplary hardware for performing the described functions is detailed below.

[0117] FIG. 6 illustrates a computer system 600 upon which an embodiment of the invention may be implemented. Although computer system 600 is depicted with respect to a particular device or equipment, it is contemplated that other devices or equipment (e.g., network elements, servers, etc.) within FIG. 6 can deploy the illustrated hardware and components of system 600. Computer system 600 is programmed (e.g., via computer program code or instructions) to an automated mashup tool as described herein and includes a communication mechanism such as a bus 610 for passing information between other internal and external components of the computer system 600. Information (also called data) is represented as a physical expression of a measurable phenomenon, typically electric voltages, but including, in other embodiments, such phenomena as magnetic, electromagnetic, pressure, chemical, biological, molecular, atomic, subatomic and quantum interactions. For example, north and south magnetic fields, or a zero and non-zero electric voltage, represent two states (0, 1) of a binary digit (bit). Other phenomena can represent digits of a higher base. A superposition of multiple simultaneous quantum states before measurement represents a quantum bit (qubit). A sequence of one or more digits constitutes digital data that is used to represent a number or code for a character. In some embodiments, information called analog data is represented by a near continuum of measurable values within a particular range. Computer system 600, or a portion thereof, constitutes a means for performing one or more steps of an automated mashup tool.

[0118] A bus 610 includes one or more parallel conductors of information so that information is transferred quickly among devices coupled to the bus 610. One or more processors 602 for processing information are coupled with the bus 610.

[0119] A processor 602 performs a set of operations on information as specified by computer program code related to an automated mashup tool. The computer program code is a set of instructions or statements providing instructions for the operation of the processor and/or the computer system to perform specified functions. The code, for example, may be written in a computer programming language that is compiled into a native instruction set of the processor. The code may also be written directly using the native instruction set (e.g., machine language). The set of operations include bringing information in from the bus 610 and placing information on the bus 610. The set of operations also typically include comparing two or more units of information, shifting positions of units of information, and combining two or more units of information, such as by addition or multiplication or logical operations like OR, exclusive OR (XOR), and AND. Each operation of the set of operations that can be performed by the processor is represented to the processor by information called instructions, such as an operation code of one or more digits. A sequence of operations to be executed by the processor 602, such as a sequence of operation codes, constitute processor instructions, also called computer system instructions or, simply, computer instructions. Processors may be implemented as mechanical, electrical, magnetic, optical, chemical or quantum components, among others, alone or in combination.

[0120] Computer system 600 also includes a memory 604 coupled to bus 610. The memory 604, such as a random access memory (RAM) or other dynamic storage device, stores information including processor instructions for an automated mashup tool. Dynamic memory allows information stored therein to be changed by the computer system 600. RAM allows a unit of information stored at a location called a memory address to be stored and retrieved independently of information at neighboring addresses. The memory 604 is also used by the processor 602 to store temporary values during execution of processor instructions. The computer system 600 also includes a read only memory (ROM) 606 or other static storage device coupled to the bus 610 for storing static information, including instructions, that is not changed by the computer system 600. Some memory is composed of volatile storage that loses the information stored thereon when power is lost. Also coupled to bus 610 is a non- volatile (persistent) storage device 608, such as a magnetic disk, optical disk or flash card, for storing information, including instructions, that persists even when the computer system 600 is turned off or otherwise loses power.

[0121] Information, including instructions for an automated mashup tool, is provided to the bus 610 for use by the processor from an external input device 612, such as a keyboard containing alphanumeric keys operated by a human user, or a sensor. A sensor detects conditions in its vicinity and transforms those detections into physical expression compatible with the measurable phenomenon used to represent information in computer system 600. Other external devices coupled to bus 610, used primarily for interacting with humans, include a display device 614, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), or plasma screen or printer for presenting text or images, and a pointing device 616, such as a mouse or a trackball or cursor direction keys, or motion sensor, for controlling a position of a small cursor image presented on the display 614 and issuing commands associated with graphical elements presented on the display 614. In some embodiments, for example, in embodiments in which the computer system 600 performs all functions automatically without human input, one or more of external input device 612, display device 614 and pointing device 616 is omitted.

[0122] In the illustrated embodiment, special purpose hardware, such as an application specific integrated circuit (ASIC) 620, is coupled to bus 610. The special purpose hardware is configured to perform operations not performed by processor 602 quickly enough for special purposes. Examples of application specific ICs include graphics accelerator cards for generating images for display 614, cryptographic boards for encrypting and decrypting messages sent over a network, speech recognition, and interfaces to special external devices, such as robotic arms and medical scanning equipment that repeatedly perform some complex sequence of operations that are more efficiently implemented in hardware.

[0123] Computer system 600 also includes one or more instances of a communications interface 670 coupled to bus 610. Communication interface 670 provides a one-way or two- way communication coupling to a variety of external devices that operate with their own processors, such as printers, scanners and external disks. In general the coupling is with a network link 678 that is connected to a local network 680 to which a variety of external devices with their own processors are connected. For example, communication interface 670 may be a parallel port or a serial port or a universal serial bus (USB) port on a personal computer. In some embodiments, communications interface 670 is an integrated services digital network (ISDN) card or a digital subscriber line (DSL) card or a telephone modem that provides an information communication connection to a corresponding type of telephone line. In some embodiments, a communication interface 670 is a cable modem that converts signals on bus 610 into signals for a communication connection over a coaxial cable or into optical signals for a communication connection over a fiber optic cable. As another example, communications interface 670 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN, such as Ethernet. Wireless links may also be implemented. For wireless links, the communications interface 670 sends or receives or both sends and receives electrical, acoustic or electromagnetic signals, including infrared and optical signals, that carry information streams, such as digital data. For example, in wireless handheld devices, such as mobile telephones like cell phones, the communications interface 670 includes a radio band electromagnetic transmitter and receiver called a radio transceiver. In certain embodiments, the communications interface 670 enables connection to the communication network 105 for an automated mashup tool for the UE 101. [0124] The term "computer-readable medium" as used herein refers to any medium that participates in providing information to processor 602, including instructions for execution. Such a medium may take many forms, including, but not limited to computer-readable storage medium (e.g., non- volatile media, volatile media), and transmission media. Non- transitory media, such as non-volatile media, include, for example, optical or magnetic disks, such as storage device 608. Volatile media include, for example, dynamic memory 604. Transmission media include, for example, coaxial cables, copper wire, fiber optic cables, and carrier waves that travel through space without wires or cables, such as acoustic waves and electromagnetic waves, including radio, optical and infrared waves. Signals include man-made transient variations in amplitude, frequency, phase, polarization or other physical properties transmitted through the transmission media. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, CDRW, DVD, any other optical medium, punch cards, paper tape, optical mark sheets, any other physical medium with patterns of holes or other optically recognizable indicia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave, or any other medium from which a computer can read. The term computer-readable storage medium is used herein to refer to any computer-readable medium except transmission media.

[0125] Logic encoded in one or more tangible media includes one or both of processor instructions on a computer-readable storage media and special purpose hardware, such as ASIC 620.

[0126] Network link 678 typically provides information communication using transmission media through one or more networks to other devices that use or process the information. For example, network link 678 may provide a connection through local network 680 to a host computer 682 or to equipment 684 operated by an Internet Service Provider (ISP). ISP equipment 684 in turn provides data communication services through the public, world-wide packet-switching communication network of networks now commonly referred to as the Internet 690. [0127] A computer called a server host 692 connected to the Internet hosts a process that provides a service in response to information received over the Internet. For example, server host 692 hosts a process that provides information representing video data for presentation at display 614. It is contemplated that the components of system 600 can be deployed in various configurations within other computer systems, e.g., host 682 and server 692.

[0128] At least some embodiments of the invention are related to the use of computer system 600 for implementing some or all of the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 600 in response to processor 602 executing one or more sequences of one or more processor instructions contained in memory 604. Such instructions, also called computer instructions, software and program code, may be read into memory 604 from another computer-readable medium such as storage device 608 or network link 678. Execution of the sequences of instructions contained in memory 604 causes processor 602 to perform one or more of the method steps described herein. In alternative embodiments, hardware, such as ASIC 620, may be used in place of or in combination with software to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware and software, unless otherwise explicitly stated herein.

[0129] The signals transmitted over network link 678 and other networks through communications interface 670, carry information to and from computer system 600. Computer system 600 can send and receive information, including program code, through the networks 680, 690 among others, through network link 678 and communications interface 670. In an example using the Internet 690, a server host 692 transmits program code for a particular application, requested by a message sent from computer 600, through Internet 690, ISP equipment 684, local network 680 and communications interface 670. The received code may be executed by processor 602 as it is received, or may be stored in memory 604 or in storage device 608 or other non-volatile storage for later execution, or both. In this manner, computer system 600 may obtain application program code in the form of signals on a carrier wave. [0130] Various forms of computer readable media may be involved in carrying one or more sequence of instructions or data or both to processor 602 for execution. For example, instructions and data may initially be carried on a magnetic disk of a remote computer such as host 682. The remote computer loads the instructions and data into its dynamic memory and sends the instructions and data over a telephone line using a modem. A modem local to the computer system 600 receives the instructions and data on a telephone line and uses an infra-red transmitter to convert the instructions and data to a signal on an infra-red carrier wave serving as the network link 678. An infrared detector serving as communications interface 670 receives the instructions and data carried in the infrared signal and places information representing the instructions and data onto bus 610. Bus 610 carries the information to memory 604 from which processor 602 retrieves and executes the instructions using some of the data sent with the instructions. The instructions and data received in memory 604 may optionally be stored on storage device 608, either before or after execution by the processor 602.

[0131] FIG. 7 illustrates a chip set 700 upon which an embodiment of the invention may be implemented. Chip set 700 is programmed to provide an automated mashup tool as described herein and includes, for instance, the processor and memory components described with respect to FIG. 6 incorporated in one or more physical packages (e.g., chips). By way of example, a physical package includes an arrangement of one or more materials, components, and/or wires on a structural assembly (e.g., a baseboard) to provide one or more characteristics such as physical strength, conservation of size, and/or limitation of electrical interaction. It is contemplated that in certain embodiments the chip set can be implemented in a single chip. Chip set 700, or a portion thereof, constitutes a means for performing one or more steps of an automated mashup tool.

[0132] In one embodiment, the chip set 700 includes a communication mechanism such as a bus 701 for passing information among the components of the chip set 700. A processor 703 has connectivity to the bus 701 to execute instructions and process information stored in, for example, a memory 705. The processor 703 may include one or more processing cores with each core configured to perform independently. A multi-core processor enables multiprocessing within a single physical package. Examples of a multi-core processor include two, four, eight, or greater numbers of processing cores. Alternatively or in addition, the processor 703 may include one or more microprocessors configured in tandem via the bus 701 to enable independent execution of instructions, pipelining, and multithreading. The processor 703 may also be accompanied with one or more specialized components to perform certain processing functions and tasks such as one or more digital signal processors (DSP) 707, or one or more application- specific integrated circuits (ASIC) 709. A DSP 707 typically is configured to process real-world signals (e.g., sound) in real time independently of the processor 703. Similarly, an ASIC 709 can be configured to performed specialized functions not easily performed by a general purposed processor. Other specialized components to aid in performing the inventive functions described herein include one or more field programmable gate arrays (FPGA) (not shown), one or more controllers (not shown), or one or more other special-purpose computer chips.

[0133] The processor 703 and accompanying components have connectivity to the memory 705 via the bus 701. The memory 705 includes both dynamic memory (e.g., RAM, magnetic disk, writable optical disk, etc.) and static memory (e.g., ROM, CD-ROM, etc.) for storing executable instructions that when executed perform the inventive steps described herein for an automated mashup tool. The memory 705 also stores the data associated with or generated by the execution of the inventive steps.

[0134] FIG. 8 is a diagram of exemplary components of a mobile terminal (e.g., handset) for communications, which is capable of operating in the system of FIG. 1, according to one embodiment. In some embodiments, mobile terminal 800, or a portion thereof, constitutes a means for performing one or more steps of an automated mashup tool. Generally, a radio receiver is often defined in terms of front-end and back-end characteristics. The front-end of the receiver encompasses all of the Radio Frequency (RF) circuitry whereas the back-end encompasses all of the base-band processing circuitry. As used in this application, the term "circuitry" refers to both: (1) hardware-only implementations (such as implementations in only analog and/or digital circuitry), and (2) to combinations of circuitry and software (and/or firmware) (such as, if applicable to the particular context, to a combination of processor(s), including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions). This definition of "circuitry" applies to all uses of this term in this application, including in any claims. As a further example, as used in this application and if applicable to the particular context, the term "circuitry" would also cover an implementation of merely a processor (or multiple processors) and its (or their) accompanying software/or firmware. The term "circuitry" would also cover if applicable to the particular context, for example, a baseband integrated circuit or applications processor integrated circuit in a mobile phone or a similar integrated circuit in a cellular network device or other network devices.

[0135] Pertinent internal components of the telephone include a Main Control Unit (MCU) 803, a Digital Signal Processor (DSP) 805, and a receiver/transmitter unit including a microphone gain control unit and a speaker gain control unit. A main display unit 807 provides a display to the user in support of various applications and mobile terminal functions that perform or support the steps of an automated mashup tool. The display 8 includes display circuitry configured to display at least a portion of a user interface of the mobile terminal (e.g., mobile telephone). Additionally, the display 807 and display circuitry are configured to facilitate user control of at least some functions of the mobile terminal. An audio function circuitry 809 includes a microphone 811 and microphone amplifier that amplifies the speech signal output from the microphone 811. The amplified speech signal output from the microphone 811 is fed to a coder/decoder (CODEC) 813.

[0136] A radio section 815 amplifies power and converts frequency in order to communicate with a base station, which is included in a mobile communication system, via antenna 817. The power amplifier (PA) 819 and the transmitter/modulation circuitry are operationally responsive to the MCU 803, with an output from the PA 819 coupled to the duplexer 821 or circulator or antenna switch, as known in the art. The PA 819 also couples to a battery interface and power control unit 820.

[0137] In use, a user of mobile terminal 801 speaks into the microphone 811 and his or her voice along with any detected background noise is converted into an analog voltage. The analog voltage is then converted into a digital signal through the Analog to Digital Converter (ADC) 823. The control unit 803 routes the digital signal into the DSP 805 for processing therein, such as speech encoding, channel encoding, encrypting, and interleaving. In one embodiment, the processed voice signals are encoded, by units not separately shown, using a cellular transmission protocol such as global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., as well as any other suitable wireless medium, e.g., microwave access (WiMAX), Long Term Evolution (LTE) networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (WiFi), satellite, and the like.

[0138] The encoded signals are then routed to an equalizer 825 for compensation of any frequency-dependent impairments that occur during transmission though the air such as phase and amplitude distortion. After equalizing the bit stream, the modulator 827 combines the signal with a RF signal generated in the RF interface 829. The modulator 827 generates a sine wave by way of frequency or phase modulation. In order to prepare the signal for transmission, an up- converter 831 combines the sine wave output from the modulator 827 with another sine wave generated by a synthesizer 833 to achieve the desired frequency of transmission. The signal is then sent through a PA 819 to increase the signal to an appropriate power level. In practical systems, the PA 819 acts as a variable gain amplifier whose gain is controlled by the DSP 805 from information received from a network base station. The signal is then filtered within the duplexer 821 and optionally sent to an antenna coupler 835 to match impedances to provide maximum power transfer. Finally, the signal is transmitted via antenna 817 to a local base station. An automatic gain control (AGC) can be supplied to control the gain of the final stages of the receiver. The signals may be forwarded from there to a remote telephone which may be another cellular telephone, other mobile phone or a land-line connected to a Public Switched Telephone Network (PSTN), or other telephony networks.

[0139] Voice signals transmitted to the mobile terminal 801 are received via antenna 817 and immediately amplified by a low noise amplifier (LNA) 837. A down-converter 839 lowers the carrier frequency while the demodulator 841 strips away the RF leaving only a digital bit stream. The signal then goes through the equalizer 825 and is processed by the DSP 805. A Digital to Analog Converter (DAC) 843 converts the signal and the resulting output is transmitted to the user through the speaker 845, all under control of a Main Control Unit (MCU) 803 -which can be implemented as a Central Processing Unit (CPU) (not shown).

[0140] The MCU 803 receives various signals including input signals from the keyboard 847. The keyboard 847 and/or the MCU 803 in combination with other user input components (e.g., the microphone 811) comprise a user interface circuitry for managing user input. The MCU 803 runs a user interface software to facilitate user control of at least some functions of the mobile terminal 801 for an automated mashup tool. The MCU 803 also delivers a display command and a switch command to the display 807 and to the speech output switching controller, respectively. Further, the MCU 803 exchanges information with the DSP 805 and can access an optionally incorporated SIM card 849 and a memory 851. In addition, the MCU 803 executes various control functions required of the terminal. The DSP 805 may, depending upon the implementation, perform any of a variety of conventional digital processing functions on the voice signals. Additionally, DSP 805 determines the background noise level of the local environment from the signals detected by microphone 811 and sets the gain of microphone 811 to a level selected to compensate for the natural tendency of the user of the mobile terminal 801.

[0141] The CODEC 813 includes the ADC 823 and DAC 843. The memory 851 stores various data including call incoming tone data and is capable of storing other data including music data received via, e.g., the global Internet. The software module could reside in RAM memory, flash memory, registers, or any other form of writable storage medium known in the art. The memory device 851 may be, but not limited to, a single memory, CD, DVD, ROM, RAM, EEPROM, optical storage, or any other non-volatile storage medium capable of storing digital data.

[0142] An optionally incorporated SIM card 849 carries, for instance, important information, such as the cellular phone number, the carrier supplying service, subscription details, and security information. The SIM card 849 serves primarily to identify the mobile terminal 801 on a radio network. The card 849 also contains a memory for storing a personal telephone number registry, text messages, and user specific mobile terminal settings.

[0143] While the invention has been described in connection with a number of embodiments and implementations, the invention is not so limited but covers various obvious modifications and equivalent arrangements, which fall within the purview of the appended claims. Although features of the invention are expressed in certain combinations among the claims, it is contemplated that these features can be arranged in any combination and order.

Claims

CLAIMS WHAT IS CLAIMED IS:

1. A method comprising:

determining a local source of functionality or data on user equipment;

causing a parameter of the local source to be presented on the user equipment; and causing determination of a second parameter of a different source to be used with the parameter of the local source in an application that serves as a new source for a network to which the user equipment can be connected.

2. A method of claim 1 , wherein determination of the second parameter of the different source further comprises:

determining of a set of one or more candidate sources based on compatibility of a plurality of sources on the network with the new source,

causing icons representing one or more of the candidate sources to be presented on the user equipment; and

determining the second parameter of the different source based on user input received in response to causing the icons to be presented.

3. A method of claim 2, wherein determining the set of one or more candidate sources further comprises determining the compatibility of the plurality of sources with the new source.

4. A method of claim 3, wherein determining the compatibility with the new source further comprises:

determining a semantic context for one or more parameters of the new source; and determining semantic contexts for published metadata about the plurality of sources on the network; and

determining compatibility based at least in part on similarity of the semantic contexts for the published metadata to the semantic context for the one or more parameters of the new source.

5. A method of claim 3, wherein determining compatibility with the new source further comprises determining compatibility based at least in part on frequency of prior use of the plurality of sources with a corresponding parameter associated with one or more parameters of the new source.

6. A method of claim 4, wherein determining compatibility with the new source further comprises determining compatibility based at least in part on frequency of prior use of the plurality of sources with a corresponding parameter associated with one or more parameters of the new source.

7. A method of claims 5 or 6, wherein the frequency of prior use comprises prior use by a user of the user equipment.

8. A method of claims 5 or 6, wherein the frequency of prior use comprises prior use by others, wherein the prior use is reported to a network service for accumulating mashup histories.

9. A method of any one of claims 1 - 6, further comprising automatically causing an icon representing the local source and an icon representing the different source to be presented as a graph on the user equipment.

10. A method of claim 9, further comprising:

determining whether the different source is a member of a set of most relevant sources; and causing the icon representing the different source to be presented on the user equipment only if the different source is determined to be a member of the set of most relevant sources.

11. A method of any one of claims 1 - 6, further comprising associating a label received as input with a parameter of the local source for determining a semantic context for the parameter.

12. A method of any one of claims 1 - 6, wherein the user equipment is a mobile terminal.

13. A method of claim 12, further comprising:

automatically causing an icon representing the local source to be presented on the user equipment;

determining whether the different source is a member of a set of most relevant sources; and causing an icon representing the different source to be presented on the user equipment only if the different source is determined to be a member of the set of most relevant sources.

14. A method of claim 12, wherein causing determination of the second parameter of the different source further comprises causing data indicating the local source to be reported to a remote server in a message using the hypertext markup language protocol and receiving the second parameter of the different source from the remote server.

15. A method of any one of claims 1-6, further comprising determining based on input from the user that the parameter of the local source and the second parameter of the different source are to be included in the new source.

16. A method of claim 15, further comprising causing a connection to be presented between a first icon representing the local source and a second icon representing the different source on a graph.

17. A method of claim 15, further comprising causing the new source that includes the parameter of the local source and the second parameter of the different source to be generated.

18. An apparatus comprising :

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the steps of a method of any one of claims 1-17.

19. An apparatus of claim 18, wherein the apparatus is a mobile phone further comprising: user interface circuitry and user interface software configured to facilitate user control of at least some functions of the mobile phone through use of a display and configured to respond to user input; and

a display and display circuitry configured to display at least a portion of a user interface of the mobile phone, the display and display circuitry configured to facilitate user control of at least some functions of the mobile phone.

20. A computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the steps of a method of any one of claims 1-17.